Tuesday, July 30, 2013

Python remote class for TAHI's v6eval engine.

The TAHI v6eval Perl module for IPv6 compliance testing at one point included a bunch of 'remote' scripts to automate tasks on the target device under test (the NUT, or node under test).

These scripts were always more like examples than set in stone automation and always required adjustment for the target device.   The scripts used a assumed serial port connection via null modem to the target and Perl expect to check for the prompt, issue commands, etc..

At work to automate the compliance testing, I wanted to use the out-of-band management port via ssh to issue commands, and to avoid long waits on reboot with just quick restarts of  services etc, so I made my own remote interface that used this:

It uses some other utils we've coded for keyswapping with the management host,  a ssh remote wrapper, etc, that I won't provide here, but you can see how it works below.

It uses v6eval's own configuration of tn.def and nut.def to provide all the connectivity,
In tn.def:


  • the 'RemoteDevice' option is re-used to provide an ssh ip address. 
  • 'RemoteMethod' is also defined to 'ssh' and although not parsed here could be easily expanded to switch different access methods based on that variable.

In nut.def:

  • the 'User' option is used to provide username for ssh access.
  • the 'Password' option is used to provide the password for ssh access.
Of course 'System' is used by the engine to determine which scripts to execute in /usr/local/v6eval/bin/[system]

All options passed to each of the '.rmt' scripts are available when using the remote class ropt dictionary for easy consumption.




#!/usr/bin/env python
"""
reboot.rmt
reboots the NUT (via a quick reload)

example invocation from the TAHI v6eval remote suite

./reboot.rmt -u root -p default -d 192.168.1.1
"""

import remote
import sys
import time

def reboot(argv):
    nut = remote.remote(argv)

    cmd = 'reload'

    rc = nut.runcmd(cmd)
    # maximum time needed for config reload
    time.sleep(90)

    if rc != 0:
        print "cmd remote command %s returned %d" % (cmd, rc)
        return nut.exitFail

    return nut.exitPass

if __name__ == '__main__':
    sys.exit(reboot(sys.argv))




# remote.py
# python v6eval remote interface class

import os
import logging
import tcutils.commandutils as cu

from tcutils import logger
from paramikoutils import keyswap

DEBUG = bool(os.environ.get('DEBUG'))
log = logger.stdout_logger('remote',
    level=logging.DEBUG if DEBUG else logging.INFO)

class remote:
    """v6eval remote interface class"""
    def __init__(self, argv):
        """init remote class"""

        # options passed by v6eval
        #my $cmd  = searchPath($rpath, $fname);
        #   $cmd .= " -t $NutDef{System}"       if $NutDef{System};
        #   $cmd .= " -u $NutDef{User}"         if $NutDef{User};
        #   $cmd .= " -p $NutDef{Password}"     if $NutDef{Password};
        #   $cmd .= " -T $TnDef{RemoteCuPath}"  if $TnDef{RemoteCuPath};
        #   $cmd .= " -d $TnDef{RemoteDevice}"  if $TnDef{RemoteDevice};
        #   $cmd .= " -v $TnDef{RemoteDebug}"   if $TnDef{RemoteDebug};
        #   $cmd .= " -i $TnDef{RemoteIntDebug}"    if $TnDef{RemoteIntDebug};
        #   $cmd .= " -o $TnDef{RemoteLog}"     if $TnDef{RemoteLog};
        #   $cmd .= " -s $TnDef{RemoteSpeed}"   if $TnDef{RemoteSpeed};
        #   $cmd .= " -l $TnDef{RemoteLogout}"  if $TnDef{RemoteLogout};
        #   $cmd .= " $RemoteOption $opts @args";

        self.exitPass=0            # PASS
        self.exitIgnore=1          # Ignore (ex. initializeation script)
        self.exitNS=2              # Not yet supported
        self.exitWarn=3            # WARN
        self.exitHostOnly=4        # Host Only
        self.exitRouterOnly=5      # Router Only
        self.exitSpecialOnly=6     # Special Only
        self.exitExceptHost=7      # Except Host
        self.exitExceptRouter=8    # Except Router
        self.exitExceptSpecial=9   # Except Special
        self.exitSkip=10           # Skip
        self.exitTypeMismatch=11   # Type Mismatch
        #
        self.exitFail=32           # FAIL
        self.exitInitFail=33       # Initialization Fail
        self.exitCleanupFail=34    # Cleanup Fail
        #                          # 35 - 63: reserved for future use
        #
        self.exitFatal=64          # FATAL (terminate series of related tests)

        self.lastrc = 0
        self.laststdout = ""
        self.laststderr = ""

        self.ropt = { }
        self.ropt = self.parseargs(argv)

        # root user
        if '-u' in argv:
            self.user = argv[argv.index('-u') + 1]
        # root password
        if '-p' in argv:
            self.passwd = argv[argv.index('-p') + 1]
        # management ip
        if '-d' in argv:
            self.mgmt_ip = argv[argv.index('-d') + 1]

        # save the original options
        self.argv = argv
        keyswap.keyswap(self.mgmt_ip, self.user, self.passwd)

    def parseargs(self, args):
        """find the foo=bar variables"""
        arg_dict = { }
        for arg in args:
            try:
                nom, val = arg.split('=')
                arg_dict[nom] = val
            except ValueError:
                pass

        return arg_dict

    def runcmd(self, cmd):
        """run the command on the NUT"""
        (self.lastrc, self.laststdout, self.laststderr) = cu.runcmd_ssh(self.mgmt_ip, cmd)
        if self.lastrc > 0:
           log.error('command: %s FAILED %s' % (cmd, self.laststderr))
           return 1
        return 0

Thursday, January 24, 2013

The sick and twisted world of KDE/konsole 4.8.4 from dbus

YUCK

For some reason, you have to find yourself when running from a konsole window now.

So the code for opening tabs now has to:

  • Grab a list of all the fricking windows (getWindowId()/getWindows() below) to find my own id.
  • Grab a list of all the fricking sessions (getSessions() below)
  • Open a new tab
  • Grab the session list AGAIN and compare it to find the one we just opened.
  • Only then are you able to send a command to that tab.
Hope you can read some shell script, because this is almost too painful to explain.

#!/usr/bin/env bash

# konsole dbus functions

getWindows() {
    windows=`qdbus org.kde.konsole /konsole org.freedesktop.DBus.Introspectable.Introspect |grep MainWindow | cut -d \" -f 2`
    echo ${windows}
}

getSessions() {
    sessions=`qdbus org.kde.konsole /Sessions org.freedesktop.DBus.Introspectable.Introspect |grep "node name" | cut -d \" -f 2`
    echo ${sessions}
}

getWindowId() {
    windows=`getWindows`
    for i in $windows
    do
        id=`qdbus org.kde.konsole /konsole/$i org.kde.KMainWindow.winId`
        if [ "${WINDOWID}" -eq "$id" ]; then
            echo $i
        fi    
    done
}    
findNewSession() {
    old="$@"
    new=`getSessions`
    for t in $new; do 
        grep -q $t <(echo $old) 
        if [ "$?" -eq "1" ]; then 
            echo $t
        fi
    done
}
sshopentabs () {
    boxlist=$1
    box_name=$2
    comm='ssh -o StrictHostKeyChecking=no -o CheckHostIP=no -o UserKnownHostsFile=/dev/null'
    
    slot=1
    for i in $boxlist
    do
        sessions=`getSessions`
        echo ${KONSOLE_DBUS_SERVICE}
        newtab
        j=`findNewSession $sessions`
        if [ "${box_name}" -eq "" ]; then
            renametab ${j} $i
        else
            renametab ${j} "${box_name} slot ${slot}"
        fi
        sendtab ${j} "${comm} root@${i}"
        let slot=$slot+1
    done

    return ${j}
}
newtab () {
    id=`getWindowId`
    dbus-send --session --dest=${KONSOLE_DBUS_SERVICE} --type=method_call --print-reply /konsole/$id org.kde.KMainWindow.activateAction string:"new-tab"
}

renametab () {
    sessionno=$1
    tabname=$2
    session="/Sessions/${sessionno}"
    dbus-send --session --dest=${KONSOLE_DBUS_SERVICE} --type=method_call --print-reply ${session} org.kde.konsole.Session.setTabTitleFormat int32:0 string:"$tabname"
    dbus-send --session --dest=${KONSOLE_DBUS_SERVICE} --type=method_call --print-reply ${session} org.kde.konsole.Session.setTabTitleFormat int32:1 string:"$tabname"
    dbus-send --session --dest=${KONSOLE_DBUS_SERVICE} --type=method_call --print-reply ${session} org.kde.konsole.Session.setTitle int32:1 string:"$tabname"
} 

sendtab () {
    sessionno=$1
    cmd=$2
    session="/Sessions/${sessionno}"
    dbus-send --session --dest=${KONSOLE_DBUS_SERVICE} --type=method_call --print-reply ${session} org.kde.konsole.Session.sendText string:"$cmd^M"
}

closetab () {
    dbus-send --session --dest=${KONSOLE_DBUS_SERVICE} --type=method_call --print-reply ${KONSOLE_DBUS_SESSION} org.kde.konsole.Session.sendText string:"exit^M"
}

Wednesday, February 15, 2012

Patch for TAHI's v6eval on FreeBSD after the switch to utmpx

After FreeBSD made the switch to utmpx, which you can read about here:

 http://thread.gmane.org/gmane.os.freebsd.current/122932

Certain programs, like the TAHI organizations v6eval package will fail to compile.   This means the recent 9.0-RELEASE of FreeBSD will no longer work out of the box.  This really only required a minor change to get it working.  I've published a patch here.

v6eval-3.3.1-utmpx.patch

Enjoy!

EDIT: ed@ kindly worked up a functional patch that works with utmpx's binary format, this might explain why sometimes your child processes are left hanging after a run:
http://80386.nl/pub/v6eval.txt

Thanks Ed!

Friday, February 10, 2012

Linux patches for TAHI v6eval 3.3.2

In my previous, ancient post running-tahi-v6eval-suite-on-linux I put out some information for posterity on converting v6eval to run on Linux systems.   Most Linux systems have moved on since three years ago and so has the v6eval suite.

I've kept up with porting the suite @$work to do automatic, unattended evaluation and reporting of our product, but I've neglected to keep it nice and clean for the rest of world.

Seeing that quite a few searches end up on that old post, I've cleaned it up and present a new patch for use against TAHI v6eval-3.3.2 .   You can download the patch HERE.

A few points about the patch and the v6eval suite:

  • To apply, extract the v6eval-3.3.2 package somewhere cd to v6eval-3.3.2 and run patch -p1 < (path to patch filel)
  • v6eval is not 64-bit clean, although it compiles fine, pktbuf will dump core.  I have not investigated or fixed this yet.   The TAHI maintainers are aware that it's not 64-bit capable.    Try to compile only in a 32bit environment.  In addition, I have not tested with -m32 compiles, if they work, let me know.
  • This compiles on a fairly recent Gentoo and a recent version of basically Centos/Redhat.(gcc 4.4.5).
  • Read the INSTALL.linux file for hints
  • Since the patch creates new files, some of these files may need permissions fixes, mainly build.sh and lorder need execute permissions.
  • Use build.sh to build, or incorporate the environment variables.
  • Getting a valid pmake is the hardest part, however Gentoo made it easy:
*  sys-devel/pmake
      Latest version available: 1.111.3.1
      Latest version installed: 1.111.3.1
      Size of files: 289 kB
      Homepage:      http://www.netbsd.org/
      Description:   BSD build tool to create programs in parallel. Debian's version of NetBSD's make
      License:       BSD
  • The patch tries to do the minimum against the distributed code to get it to work with one exception.
  • The patch contains my custom pcap filter enhancement for the tn.def file:
Normally the engine only supports filtering for just ipv6 packets with a single line in the tn.def file:

filter ipv6


Since the automation harness at work has a busy network, I've modified the filter expression to be capable of any valid bpf filter, which means you need use similar syntax to tcpdump, so
filter ipv6
becomes
filter ip6
.

I use something similar to the following example to restrict tests to only the NUT, using the NUT's mac address:

filter ether host aa:bb:cc:dd:ee:ff and ip6


REF:
http://dl.dropbox.com/u/18896363/v6eval-3.3.2-linux.patch

Monday, September 12, 2011

pidgin sipe win32 compiles w/ pidgin 2.10 (libpurple 2.10.0 & sipe-1.12.0)

UPDATE: The sipe project appears to have official builds now:

http://sourceforge.net/projects/sipe/files/sipe/

UPDATE/EDIT: As some of you have figured out from the config.h blurb below, this is actually pidgin-sipe 1.12.0.  I just re-upped to dropbox with a name change on the .zip.   I guess I was trying to brain dump to this blog too fast so I wouldn't forget the steps to compiling it.

Well I did it.



What  a mess it is to compile some of these programs on win32.

I followed the basic instructions on compile pidgin and got that all working.    These instructions are actually pretty good.  They are begging to be automated similar to a FreeBSD port build:

http://developer.pidgin.im/wiki/BuildingWinPidgin

Then I tried to follow the pidgin sipe wiki entry for windows:

http://sourceforge.net/apps/mediawiki/sipe/index.php?title=Windows_Build

These are a bit out of date.   The good thing is that if you're building post 2.7.0 then most dependencies are already there.

There were a couple things broken at this point preventing success:

  • one file was missing an include for glib.h -- already fixed in the sipe git rep
  • you need gmime installed in your pidgin-devel/win32-dev directory (linked on the sipe wiki)
  • you need the rest of libiconv installed in your pidgin-devel/win32-dev directory (grab libiconv-1.13.1-1-mingw32-dev.tar.lzma from the mingw site)
  • libxml2 needed the libs renamed to libxml2-2.lib in their win32-dev directory.
  • sipe-core.c wouldn't compile with several entries from config.h which are only generated when running configure.   Since running configure correctly in this setup would be a veritable pain, I managed to get what it wanted figured out:
config.h

#define PACKAGE_NAME "pidgin-sipe"
#define PACKAGE_TARNAME "pidgin-sipe"
#define PACKAGE_VERSION "1.12.0"
#define PACKAGE_STRING "pidgin-sipe 1.12.0"
#define PACKAGE_BUGREPORT "https://sourceforge.net/tracker/?atid=949931&group_id=194563"
#define PACKAGE_URL "http://sipe.sourceforge.net/"
#define SIPE_TRANSLATIONS_URL "https://www.transifex.net/projects/p/pidgin-sipe/r/mob/"



Save this as config.h and plop it in src/core and delete the #ifdef HAVE_CONFIG_H stanza in sipe-core.c to allow it to be included.  This will allow sipe_core_about() to compile.

Successful compile.   Yay.

After this I copied libxml2-2.dll and libiconv-2.dll to my pidgin-2.10.0/win32-install-dir directory and src/core/libsipe.dll to the win32-install-dir/plugins directory.   Viola, it worked.

You can download it here win32 libpurple 2.10.0 pidgin-sipe-1.12.0

Thursday, July 7, 2011

moved from livejournal...

Not sure that it requires any explanation (all you have to do is look at livejournal's front page).   


Used google-blog-converters to grab all the posts and convert them to blogger.xml.   Then used the import feature here on the settings tab.   Worked great!

I then promptly deleted my livejournal account.

Monday, April 11, 2011

konsole tabs with dbus in kde4.6

In the continuing automation saga with konsole at my job, KDE has come far enough with the dbus interface in 4.6 to allow me to bring up tabs again and perform actions in them with a script.

For this I created the following shell functions:


# konsole dbus functions
newtab()
{
dbus-send --session --dest=${KONSOLE_DBUS_SERVICE} --type=method_call --print-reply /konsole/MainWindow_1 org.kde.KMainWindow.activateAction string:"new-tab"
}

renametab()
{
sessionno=$1
tabname=$2
session="/Sessions/${sessionno}"
dbus-send --session --dest=${KONSOLE_DBUS_SERVICE} --type=method_call --print-reply ${session} org.kde.konsole.Session.setTabTitleFormat int32:0 string:"$tabname"
dbus-send --session --dest=${KONSOLE_DBUS_SERVICE} --type=method_call --print-reply ${session} org.kde.konsole.Session.setTabTitleFormat int32:1 string:"$tabname"
dbus-send --session --dest=${KONSOLE_DBUS_SERVICE} --type=method_call --print-reply ${session} org.kde.konsole.Session.setTitle int32:1 string:"$tabname"
}

sendtab()
{
sessionno=$1
cmd=$2
session="/Sessions/${sessionno}"
" dbus-send --session --dest=${KONSOLE_DBUS_SERVICE} --type=method_call --print-reply ${session} org.kde.konsole.Session.sendText string:"$cmd
}

closetab()
{
dbus-send --session --dest=${KONSOLE_DBUS_SERVICE} --type=method_call --print-reply ${KONSOLE_DBUS_SESSION} org.kde.konsole.Session.close
}



In scripts that I want automation, I include these functions:


# konsole functions
. ~/bin/kons-fn.sh



I can then open a whole bunch of tabs at once and send them a command:



PORTS=( 2022 2021 )

SESSIONS="
A_FU
B_BAR
"

command="telnet ${CONSOLE_SERVER}"

j=1
index=0
for i in ${SESSIONS}
do
newtab
let index=$j-1
let j=$j+1
renametab $j $i
sendtab $j "${command} ${PORTS[${index}]}"
done

closetab
exit 0



However I haven't gotten this to work from starting 'konsole -script' from a shortcut. Launching a script like this creates an infinite number of tabs launching the script in sequence. Very weird stuff.

For now, just start a konsole session and launch the script within the first tab. As you can deduce from the functions, it has to be the first tab you launch konsole with.