Convert MAC addresses from hex to dec and dec to hex

In my work I frequently deal with MAC addresses. I'll get hex MAC addresses from routers and then need to look them up in a database where the are stored in MySQL as big integers.

When I do my lookups I use SqlYOG running in wine. I when I formulate my queries I can use MySQL's 0x100000000001 syntax if it's just one MAC address. If there are more MAC addresses I can use a simple set of shell functions defined in a shell library file and sourced in my .bashrc to convert them for me.

h2d(){
  mac=$(echo $1 |  tr '[:lower:]' '[:upper:]')
  echo "ibase=16; ${mac}"|bc
}
d2h(){
  mac=$(echo $1 |  tr '[:lower:]' '[:upper:]')
  echo "obase=16; ${mac}"|bc
}

dhall(){
    for ii in $*
    do
        printf "%012X " $ii
    done
    echo
}

hdall(){
    for ii in $*
    do
        h='0x'$(echo $ii | tr '[:lower:]' '[:upper:]')
        printf "%d "  $h
    done
    echo
}

hdall converts hex numbers to base 10 and dhall converts a base 10 integer to hex. This is handy at the shell in case I'm using the MySQL command line utility.

I've included some older variants where I used bc as my converter. You'll find that the printf versions are much faster, however, not all unix shells or unix flavors offer bash and printf. You may need the bc versions for those.

Also notice that I move all of the hex characters to uppercase for conversion. The bc utility won't take the alpha characters if they are in lower case.

Let's say I have the following for integer MAC addresses someone gives me.

207299441470 207299462794 207299730467442 207299474  

I can use dhall to convert them to hex and I can use hdall to convert them back. Using printf it's easy to pad the hex numbers out to the correct length.

[gail@gail]$ dhall 7299441470 207299462794 207299730467442 207299474  
0001B314A33E 00304402C68A BC89BACBEA72 00000C5B2392 

[gail@gail]$ hdall 0001B314A33E 00304402C68A BC89BACBEA72 00000C5B2392 
7299441470 207299462794 207299730467442 207299474 

Another way to run scripts from vim

In a previous post I presented a shell script that you can call from vim using a key mapping that would run the current script you are working on in your editor session with any command line arguments specified. The script looked at the file extension and called the appropriate interpreter.

Look Here!

This is a very handy method, however, many times an executable script utility doesn't have a file extension. Without the file extension the previously present script won't work. To deal with this case, I've created another one that can be used either instead of the first or along with the first to get both cases.

My new script uses the shebang line of the script to get the interpreter. That gives me a way to launch a script utility using a key mapping in vim but I don't have to have a file extension on the file.

The script listing follows:

#!/bin/bash

# Make sure we have at least one argument
if [[ -z $1 ]]
then
    exit 1
fi

# Save our script name
script=$1

# Get the interpreter from the shebang line
interpreter=$(grep '#!/' $script | sed -r 's/(^.*?#!)//')

# Remove the first argumen
shift
args=$@

$interpreter $script $args

This script calls on a simple grep mechanism to grab the shebang line, take off the grep output with sed, to create an absolute path to an appropriate interpreter.

Take note of the '-r' option to sed. This enables us to use extended regular expressions. Your linux distro may have a different implementation of sed, so make sure to test this before you use it.

All the best.

An easy way to remember what your scripts do

I have a lot of shell utilities in my home bin directory. Some of them I use all of the time and know what they are. Others, no matter how cleverly I name them, I'll not use for long periods of time and will forget what they are for. I can use -h, -H, or --help on each one individually but that takes a lot of time to find the one I'm looking for.

In addition to just you looking for utilities you may create utilities that go into a shared directory resource and others will want to know what the scripts do without having to run help or man on each.

In order to deal with this I created a very simple shell utility called "synopsis". It's little more than a grep line with some sed filtering to get a formatted comment out of my files that tells me what they do. That makes it easier and faster to find what I'm looking for and then I can go into detail by looking at m help and usage using the script's tools, or go in and look at the code and internal comments.

Here is an example of synopsis' output:

[gail@gail]$ synopsis

buildterm:       Launch a stylized xterm that identifies it as my build terminal
evar:    Print formatted output of the environment variables passed as args
foo.php:         Get the target webpage and redirect the output to a blog entry
path:    Prints a one per line list of the $PATH elements.

You can do more to columnize the output, however, I find the quick reminder is good enough in the form that it is.

This script could actually be an alias however, I like having the ability to do more with the formatting if I choose.

This is the script listing:

#!/bin/bash

echo 
grep 'Synopsis:' * | grep -v 'grep Synopsis' |  awk 'BEGIN { FS = ":" } ; { printf "%-30s%-40s\n",$1,$3 }'  -
echo 

As you can see it's very simple. We use grep the find the hatch comment indicator followed by the string 'Synopsis:'. Then we use grep -v to strip out our synopsis script line by looking for 'grep Synopsis'. Grep will give us the script name and colon automatically and awk's file separator will let us ignore the '#Synopsis:' string. We use printf to do a little pretty printing of the strings.

You do have to take some care to keep your synopsis text on one line, however, it will be the most common case for you to want to keep it short anyway.

Most scripting languages recognize the '#' character as a comment so this should work all through your interpreter source trees. If you want it to recognize C++ and javascript comments you can use egrep versus grep to find the '//' comment specifier and then follow the same filtering rules.

If you wanted to extend this idea further you can use perl to look for the string Synopsis: in all of your files. That will give you better control over the look of the output and it will allow you to use synopsis for any type of source file using any type of comment specifier native to that language.

All the best.

A simple way to view shell environment variables

Most of the time when you deploy or run a bigger application or writing a complex program you'll find you have problems with your environment. You'll find everything from environment variables not set or not set correctly to missing resources.

In order to quickly check the settings of environment variables I've created a simple python shell utility to do that for me.

I called the utility evar for "environment variable". It lets me put a list of environment variable behind the script name and have my environment checked for the presence of each of them in by upper and lower case forms.

Example:

[gail@gail]$ evar home AFWE qwer foo 

HOME                           /home/gail                              
AFWE                           IS NOT SET                              
QWER                           IS NOT SET                              
FOO                            FOOOOO                                  
foo                            bar

Additionally, when we get long path listings it can be difficult to find the element we are searching for with all of the elements on one line. So in the script I check for the presence of colons ':' in the line then format the print output so that there is one element to a line. This breaks up the path so that it can be more easily read and make it easier to find the elements we are looking for.

That means that two variables defined as:

[gail@gail]$ echo $FOO
FOOOOO

[gail@gail]$ echo $foo
asdf:qwerqwer:zxcvzxcv

Will look like this when formatted by evar:

[gail@gail]$ evar foo

FOO                            FOOOOO                                  
foo
                               asdf                                    
                               qwerqwer                                
                               zxcvzxcv 

The script's listing follows. You can do what you what you want with the formatting. I like the breathing room and the ordered lay out.

#!/usr/bin/python

import os, sys, re
print 
for ii in sys.argv[1:]:
        var = ii.upper()

        if var not in os.environ and ii not in os.environ:
                print "%-30s %-40s" % (var,'IS NOT SET')

        if var in os.environ:
                mvar = os.environ[var]
                if re.search(':',mvar):
                        print var
                        for val in mvar.split(':'):
                                print "%-30s %-40s" % ('',val)
                else:
                        pvar = os.environ[var]
                        print "%-30s %-40s" % (var,pvar)


        if ii in os.environ:
                mvar = os.environ[ii]
                if re.search(':',mvar) is not None:
                        print ii
                        for val in mvar.split(':'):
                                print "%-30s %-40s" % ('',val)
                else:
                        pvar = os.environ[ii]
                        print "%-30s %-40s" % (ii,pvar)


print 

A better way to lauch code from vim or a text editor

In a previous post I wrote about a simple vim map to launch the code I'm currently working on in an editor session without leaving the current file or without opening another window or tab in your terminal emulator. (See http://sphyrnalewini.blogspot.com/2012/02/another-nice-vim-trick-for-multi.html)

As I add plugins and shortcuts to vim and add the need to setup more complex environments to run my scripts I find that using the map on multiple keys doesn't work anymore. Plugins consume FKey mappings and sometimes you need to have path and config variables set in order to run your code.

To handle this I've written a simple shell script that takes the output of the same vim mapping syntax

:w |!interpreter_name %

And lets you run all of the interpreters from the one script. This reduces the number of key combinations you have to use for programs and also lets you do more with the environment around the script.

#!/bin/bash
# Make sure we have at least one argument
if [[ -z $1 ]]
then
 exit 1
fi

# Save our script name
script=$1

# Get the extension
extension=$(echo $1 | sed 's/\(.*\)\.//' )

# Remove the first argument
shift

# Create the argument string
args=$@

# Go through our list of extensions and
# call the appropriate interpreter.
# We can also add command line args to
# the interpreter or do redirect statements.
case $extension in
 py)
  # Add an environment varialbe for the  script.
  export FOO='YOUFOUNDFOO'
  /usr/bin/python $script $args
  ;;
 php)
  /usr/bin/php $script $args
  ;;
 rb)
  /usr/bin/ruby $script $args
  ;;
 pl)
  /usr/bin/perl $script $args
  ;;
 sh)
  /bin/bash $script $args
  ;;
 *)
  echo OOOPS $script $args
  exit 1
  ;;
esac

Notice in the python case the environment variable 'FOO'. This could be simple things like paths or maybe linking strings or paths to python resources you need for this script or it could be as complex as setting up a full Django environment and logging into and out of virtualenv.

Then if you run the following simple python script:

import os

print os.getenv('FOO')
print 'foobar'

You'll see the value of the environment variable 'FOO' 'YOUFOUNDFOO'.

A more practical use might be to include a remote system path that has resources the script needs like testing code for pytest or nose as in the example below.

# in runme shell script
py)
  export RESOURCEDIR="/home/dev/libs/python/mylibdir"

Then in your python script:

import sys
import os
sys.path.append(os.getenv('RESOURCEDIR'))

Remember that you can also use special command line args from vim to add these as well, because your python script can simply ignore any positional args. They would be strictly for use in the shell script.

Using the practice outlined with this script it would be easy enough to create more complex handling of individual scripts or change the behavior based on what the current directory is that you are working in.

All the best.

Crunchbang Linux

I love to try new linux distributions. I have both very capable hardware and some older hardware that is good but won't deal with an OS that puts an emphasis on animation eye candy or huge configurations that live in memory.

One of my favorite new distros is #!linux (Crunchbang http://crunchbanglinux.org/). It's very light, but at the same time has some of the nicer user features Fedora, Ubuntu or Mint users have grown accustomed to.

It's not as light as say Arch Linux or Puppy Linux, but it definitely has more usability features than those two very lean distros. I like Openbox a lot and when it comes to doing work quickly it gives you want you want all at the right click of your mouse.

I think that one of the best features is that it has very up to date software repositories and uses aptitude and synaptic as a package manager. If you are coming from some Redhat variant like Fedora or CentOS then you'll find the apt commands have a good YUM feel. If you are an rpm CLI sort then debian variants give you dpkg that works and behaves very rpm like.

There are many good reviews of Crunchbang so I'm not going to go into that level of detail here. I do want to call your attention to it and if you find it difficult to deal with the new permutations of Ubuntu or Gnome 3 then I think you'll find Crunchbang a refreshing change.

All the best.

Another nice vim trick for multi language folks

I work with several languages on a daily basis. Because I work in vim most of the time and I'm working with interpreted languages I Used to have to either get out of the editor or have another window open (or tab in mrxvt or terminator) in order to run the files to test changes as I make them. This simple vim trick eliminates the need for either of those solutions.

" run python scripts
map :w\|!python %
" run php scripts
map :w\|!php %
" run ruby scripts
map :w\|!ruby %
" run perl scripts
map :w\|!perl %
" run make in the current directory
map :w\|!make

By adding these lines to your .vimrc you can easily run the current script, see the output, and then with any keystroke return to the exact spoy where you where typing.

I write the file file first in the map but you don't have to. Remember that if you don't and you don't write out your changes yourself the script won't do anything any differently than it did before.

If you want command line arguments all you have to do is add them at the vim command prompt. Let's say you run a python script and are adding command line switches and arguments to it. You press F2 and it runs your script without any arguments. Then you add argument code and want to see the effect. So in command mode you press ":" (colon) and then up arrow to get the last command that will be:

:w\|!python %

All you do is add your arguments to the end of this line as in:

:w\|!python % --XYZ-X=/foo/bar -d --quiet

python will run your scripts just as though you were on the command line.

There are other ways of doing this, but I've found this one to be the most useful. When you get into combining makefiles with shell scripts wrapped around interpreter commands it can be much more powerful.

All the best.