Sunday, February 5, 2012

A simple way to view shell environment variables

0 comments
Most of the time when you deploy or run a bigger application or writing a complex program you'll find you have problems with your environment. You'll find everything from environment variables not set or not set correctly to missing resources.

In order to quickly check the settings of environment variables I've created a simple python shell utility to do that for me.

I called the utility evar for "environment variable". It lets me put a list of environment variable behind the script name and have my environment checked for the presence of each of them in by upper and lower case forms.

Example:
[gail@gail]$ evar home AFWE qwer foo 

HOME                           /home/gail                              
AFWE                           IS NOT SET                              
QWER                           IS NOT SET                              
FOO                            FOOOOO                                  
foo                            bar

Additionally, when we get long path listings it can be difficult to find the element we are searching for with all of the elements on one line. So in the script I check for the presence of colons ':' in the line then format the print output so that there is one element to a line. This breaks up the path so that it can be more easily read and make it easier to find the elements we are looking for.

That means that two variables defined as:

[gail@gail]$ echo $FOO
FOOOOO

[gail@gail]$ echo $foo
asdf:qwerqwer:zxcvzxcv

Will look like this when formatted by evar:

[gail@gail]$ evar foo

FOO                            FOOOOO                                  
foo
                               asdf                                    
                               qwerqwer                                
                               zxcvzxcv 

The script's listing follows. You can do what you what you want with the formatting. I like the breathing room and the ordered lay out.

#!/usr/bin/python

import os, sys, re
print 
for ii in sys.argv[1:]:
        var = ii.upper()

        if var not in os.environ and ii not in os.environ:
                print "%-30s %-40s" % (var,'IS NOT SET')

        if var in os.environ:
                mvar = os.environ[var]
                if re.search(':',mvar):
                        print var
                        for val in mvar.split(':'):
                                print "%-30s %-40s" % ('',val)
                else:
                        pvar = os.environ[var]
                        print "%-30s %-40s" % (var,pvar)


        if ii in os.environ:
                mvar = os.environ[ii]
                if re.search(':',mvar) is not None:
                        print ii
                        for val in mvar.split(':'):
                                print "%-30s %-40s" % ('',val)
                else:
                        pvar = os.environ[ii]
                        print "%-30s %-40s" % (ii,pvar)


print 

A better way to lauch code from vim or a text editor

0 comments
In a previous post I wrote about a simple vim map to launch the code I'm currently working on in an editor session without leaving the current file or without opening another window or tab in your terminal emulator. (See http://sphyrnalewini.blogspot.com/2012/02/another-nice-vim-trick-for-multi.html)

As I add plugins and shortcuts to vim and add the need to setup more complex environments to run my scripts I find that using the map on multiple keys doesn't work anymore. Plugins consume FKey mappings and sometimes you need to have path and config variables set in order to run your code.

To handle this I've written a simple shell script that takes the output of the same vim mapping syntax

:w |!interpreter_name %

And lets you run all of the interpreters from the one script. This reduces the number of key combinations you have to use for programs and also lets you do more with the environment around the script.

#!/bin/bash
# Make sure we have at least one argument
if [[ -z $1 ]]
then
 exit 1
fi

# Save our script name
script=$1

# Get the extension
extension=$(echo $1 | sed 's/\(.*\)\.//' )

# Remove the first argument
shift

# Create the argument string
args=$@

# Go through our list of extensions and
# call the appropriate interpreter.
# We can also add command line args to
# the interpreter or do redirect statements.
case $extension in
 py)
  # Add an environment varialbe for the  script.
  export FOO='YOUFOUNDFOO'
  /usr/bin/python $script $args
  ;;
 php)
  /usr/bin/php $script $args
  ;;
 rb)
  /usr/bin/ruby $script $args
  ;;
 pl)
  /usr/bin/perl $script $args
  ;;
 sh)
  /bin/bash $script $args
  ;;
 *)
  echo OOOPS $script $args
  exit 1
  ;;
esac

Notice in the python case the environment variable 'FOO'. This could be simple things like paths or maybe linking strings or paths to python resources you need for this script or it could be as complex as setting up a full Django environment and logging into and out of virtualenv.

Then if you run the following simple python script:

import os

print os.getenv('FOO')
print 'foobar'

You'll see the value of the environment variable 'FOO' 'YOUFOUNDFOO'.

A more practical use might be to include a remote system path that has resources the script needs like testing code for pytest or nose as in the example below.

# in runme shell script
py)
  export RESOURCEDIR="/home/dev/libs/python/mylibdir"

Then in your python script:

import sys
import os
sys.path.append(os.getenv('RESOURCEDIR'))

Remember that you can also use special command line args from vim to add these as well, because your python script can simply ignore any positional args. They would be strictly for use in the shell script.

Using the practice outlined with this script it would be easy enough to create more complex handling of individual scripts or change the behavior based on what the current directory is that you are working in.

All the best.

Crunchbang Linux

0 comments
I love to try new linux distributions. I have both very capable hardware and some older hardware that is good but won't deal with an OS that puts an emphasis on animation eye candy or huge configurations that live in memory.

One of my favorite new distros is #!linux (Crunchbang http://crunchbanglinux.org/). It's very light, but at the same time has some of the nicer user features Fedora, Ubuntu or Mint users have grown accustomed to.

It's not as light as say Arch Linux or Puppy Linux, but it definitely has more usability features than those two very lean distros. I like Openbox a lot and when it comes to doing work quickly it gives you want you want all at the right click of your mouse.

I think that one of the best features is that it has very up to date software repositories and uses aptitude and synaptic as a package manager. If you are coming from some Redhat variant like Fedora or CentOS then you'll find the apt commands have a good YUM feel. If you are an rpm CLI sort then debian variants give you dpkg that works and behaves very rpm like.

There are many good reviews of Crunchbang so I'm not going to go into that level of detail here. I do want to call your attention to it and if you find it difficult to deal with the new permutations of Ubuntu or Gnome 3 then I think you'll find Crunchbang a refreshing change.

All the best.

Another nice vim trick for multi language folks

0 comments
I work with several languages on a daily basis. Because I work in vim most of the time and I'm working with interpreted languages I Used to have to either get out of the editor or have another window open (or tab in mrxvt or terminator) in order to run the files to test changes as I make them. This simple vim trick eliminates the need for either of those solutions.

" run python scripts
map :w\|!python %
" run php scripts
map :w\|!php %
" run ruby scripts
map :w\|!ruby %
" run perl scripts
map :w\|!perl %
" run make in the current directory
map :w\|!make

By adding these lines to your .vimrc you can easily run the current script, see the output, and then with any keystroke return to the exact spoy where you where typing.

I write the file file first in the map but you don't have to. Remember that if you don't and you don't write out your changes yourself the script won't do anything any differently than it did before.

If you want command line arguments all you have to do is add them at the vim command prompt. Let's say you run a python script and are adding command line switches and arguments to it. You press F2 and it runs your script without any arguments. Then you add argument code and want to see the effect. So in command mode you press ":" (colon) and then up arrow to get the last command that will be:

:w\|!python %

All you do is add your arguments to the end of this line as in:

:w\|!python % --XYZ-X=/foo/bar -d --quiet

python will run your scripts just as though you were on the command line.

There are other ways of doing this, but I've found this one to be the most useful. When you get into combining makefiles with shell scripts wrapped around interpreter commands it can be much more powerful.

All the best.

A couple of nice VIM tricks

0 comments
I really like VIM. I'm used to it and it's fast, flexible, and powerful. Once you learn how to extended it VIM opens up many new avenues to productivity.

This is a really basic trick but it's powerful. It's one of those that I spent a long time wishing vim would do only to learn that "IT COULD" do exactly what I wanted it to do!

One of the complaints about console text editors is that they don't scroll using the mouse wheel. VIM is not one of those. If you add this to your .vimrc file:

if has('mouse')
set mouse=a
endif

Vim will use the mouse wheel to let you scroll through the file you are working on!

If you code in python these next two tricks will be a big help. I work on all sorts of projects and once in awhile I'll get a file that uses spaces for tabs. I'm currently using tabs for everything because that's the standard where I work. But if I pick up a PEP8 compliant file then I'll get spaces. So to do a quick conversion while I'm in the file I put the following settings in my .vimrc:

map :%s/ \{4}/ /g

That will convert all spaces to tabs in command mode. Just reverse it or use another f key or key combination if you want to go from tabs to spaces.

This last trick is especially for python folks. Sometimes extra spaces will creep in between some text and your tabs. The interpreter doesn't like this at all and will make your life difficult by not compiling your file. If you have a space before a tab you can use this vim trick to highlight the spaces so that you can easily find them.

"Mark trailing whitespace and spaces before tabs
highlight RedundantSpaces term=standout ctermbg=red guibg=red
match RedundantSpaces /\s\+$\| \+\ze\t/ "\ze sets end of match so only spaces highlighted
set listchars=tab:>-,trail:.,extends:>

Be a little careful with this one because some themes you choose or syntax files will make this silently fail.

All the best.

Monday, March 28, 2011

fping -- A nice addition to the networking toolbox

0 comments
I work in a networking environment and so I've been looking for tools to do quick jobs without having to go out and get an SNMP setup going. One of the tasks I have is occasionally tracking down a device or router by it's MAC address.

The great network utility nmap will do it but for a quick job I found fping http://fping.sourceforge.net/.

In order to find out the IP of an errant router by its MAC address you use arp -a or arp -n to look in your ARP cache. In order to get it in there you can write a script to ping every host in a subnet or you can use fping.

$ fping -c 1 192.168.0.0/24

This will rapidly ping the subnet filling the arp cache with the IP to MAC address mappings.

Then you can use (depending on your arp implementation, mine is a Debian variant)

$ arp -n

To print out a list of IP's and MAC addresses.

You may have to use

$ arp -a

To make things go a little faster if you have a big list you can grep for the last few digits of the MAC

arp -n (or a) |grep -i 863b2

You'll get your IP and you won't have to go sniffing with the very capable but complex tool wireshark or some similar.

All the best!

Saturday, November 6, 2010

VirtualBox

0 comments
It's difficult to be around tech these days and not have heard about virtualization. VirtualBox http://www.virtualbox.org/ is one of the products that brings virtualization to the desktop.

I used to use VMware player. Once I was introduced to VirtualBox I switched all of my systems to use it. It's easy to install and manage and you get a great range of capabilities with the one product. VirtualBox has a great community of users that makes getting support relatively easy. It runs on most all popular platforms and extends your capabilities as a tech professional.

I run Ubuntu 10.04 LTS as my host system with Virtualbox running Win 7, two installs of Win XP pro, Fedora 14, CentOS6, Linux Mint, Open SUSE, and most recently Free BSD. I use these virtual machines to test scripts, applications, and browser comparability with web application that I'm developing. I also use them to develop and test platform specific deployments of server side products.

In addition to test environments I use virtual machines to give me some capability that another OS or distribution can't, and I'll just try out some new tech toy or tool that I couldn't do easily without having access to a full boot partition or even another box. For instance, I'm trying out Linux Mint with XFCE.

Coupling all of the base capability to the great tools the VirtualBox folks deliver in the guest additions and then capping it off with seamless mode the software opens up many learning and professional doors that would be difficult or expensive without access to the virtual machine.

Give it a try. I'm sure you'll get a kick out of it!