Wednesday, June 10, 2009

HOTO: Create html "snapshot" from your gvim editor

Here is my .vimrc snippet that allows me to quickly save off a file in an HTML format with all of the syntax highlighting, etc..

The function is not working.. :( will have to work on it more later...


" Settings for :TOhtml
let html_number_lines=1
let html_use_css=1
let use_xhtml=1

""function not working
"function! MyToHtml()
" normal :colorscheme default
" normal :TOhtml
" normal :colorscheme torte
"endfunction
"nmap <Leader>html :call MyToHtml()
nmap <Leader>html :TOhtml<CR>zR :w! ~/tmp/<C-R>%<CR> :q!<CR>
nmap <Leader>print :colorscheme default<CR> :TOhtml<CR>zR :w! ~/tmp/<C-R>%<CR> :q!<CR>zR:colorscheme torte<CR>

Monday, June 8, 2009

COLRM - Discovered a fun new tool!

It seems amazing to me that no matter how long I work in the world of Linux I am still to this day finding little tools to help my productivity. There are many different ways to remove certain columns of information from a text file but I just discovered colrm ( colrm first_col [second_col] ) which is a tool to do specifically that task. I am not sure how it does this behind the scenes so I am not sure how well it works if performance is critical but for a hackey little side thing it seems to do very well.

Here is an example where I removed columns 87 to the end and from column 1 to 69 leaving just column data from 70 to 86:


> cat ~/tmp/max_data.out colrm 87 colrm 1 69 > ~/tmp/max_data2.out

Friday, June 5, 2009

A better unit conversion script (h2b b2h h2d d2h)

So I have been trying to get a robust unit conversion script for a while and I have had a few different solutions but I always had some troubles with them. bc is the calculator tool that comes with most linux distributions and is pretty widely available and is very good at doing the conversions but it too had some difficulties if the string containing the numbers was not formatted just right. So I use a small Perl wrapper script to do the formatting and then pass it on to bc. This seems to be working very well for me now. You can see the code below.

I also have some vim commands that call this script and allow me to do do conversions right from within vim (will add this vim code later). I also have some csh alias commands that I will add later as well.

It has proven to be very powerful and has improved my coding and debugging efficiency. Hope someone else finds it useful.


##### FILE: bc_conversion.pl ######


#!/usr/local/bin/perl -w
# Use the bc function to do conversions..
#
## | check if called properly
#if ($#ARGV >= 0) {
if ($#ARGV < 2) {
print " Usage: $0 \n" ;
print " used to wrap bc so you can set the base input and output \n" ;
print " in order to do conversions. For example to convert binary \n" ;
print " to hex, run the following command: \n" ;
print " bc_wrapper.sh 2 16 1001_1011 \n" ;
print " RESULT: 9B \n" ;
exit
}

$ARGV[2] =~ s/_//g;
$ARGV[2] =~ tr/a-z/A-Z/;

# The following is a csh command that tells the tool bc the output base type,
# the input base type and the expression to run.
my $cmd = "bc << EOF \
obase=$ARGV[1]; \
ibase=$ARGV[0]; \
$ARGV[2]; \
quit; \
EOF";

#print " $cmd \n";
#system("$cmd") == 0 or die "system $cmd failed: $?";
my $result = `$cmd`;
# Make the result easier to read by inserting underscores
if ( ( $ARGV[1] eq "2" ) || ( $ARGV[1] eq "16" ) ) {
$result = reverse $result;
$result =~ s/(\w{4})/$1_/g;
$result = reverse $result;
$result =~ s/^_//g;
}
print "$result";

My faster little find script

I have been using this script for a long time. It certainly could be improved but it has proved invaluable.

##### FILE: find.pl #####


#!/usr/local/bin/perl -w
use strict;
use Getopt::Long;
use Pod::Usage;

my $cmdLine = "rerunRegressionResults.pl @ARGV \n";

my $dateStamp=`date +'%Y-%m-%d-%H%M'`; chomp($dateStamp);
my $output = "";
my $help = 0;
my $grep = "";
my $displayOnly = 0;
my $noGrep = 0;
my $noVim = 0;
my $vimCmd = "";
my $dryRun = 0;
my $cmd = "";
my @fileNames;
my $replaceWithSpace = "~"; # Default is to replace all ~ with spaces.
my $arg = "";
my $verbosity = "";
my $endFind = "";
my @strings;
my $grepArg = "";
my $findRslt = "";
my $findRsltFound = 0;
my $endGrep = "";
my @dirs;
my @findRslts;
my @nots;
my @ands;
my @prunes;
my @sqlResults;
my @outputList;

my $ok = GetOptions(
'displayOnly|do' => \$displayOnly,
'noVim|nv' => \$noVim,
'vimCmd|vc=s' => \$vimCmd,
'dryRun|dr' => \$dryRun,
'dirs=s@' => \@dirs,
'output=s' => \$output,
'endGrep|eg=s' => \$endGrep,
'fileNames|fn=s@' => \@fileNames,
'noGrep|ng' => \$noGrep,
'arg=s' => \$arg,
'verbosity=s' => \$verbosity,
'endFind|ef=s' => \$endFind,
'strings=s@' => \@strings,
'grepArg|ga=s' => \$grepArg,
'nots=s@' => \@nots,
'ands=s@' => \@ands,
'prunes=s@' => \@prunes,
'replaceWithSpace=s' => \$replaceWithSpace,
'help' => \$help
);
if ( !$ok ) { die; }

pod2usage(1) if $help;

push ( @strings , @ARGV );
if ( ( @strings == 0 ) && ( @fileNames == 0 ) ) { die " NO SEARCH STRING OR FILE NAME GIVEN!!! "; }

# Decide the defaults:
if ( @fileNames == 0 ) {
$fileNames[0] = "*";
} else {
if ( @strings == 0 ) { $strings[0] = $fileNames[0]; }
}
if ( @dirs == 0 ) {.
$dirs[0] = "$ENV{'PWD'}";.
}
if ( $vimCmd eq "" ) { $vimCmd = "/usr/local/bin/gvim"; }
if ( $arg eq "" ) { $arg = ""; }
if ( $endFind eq "" ) { $endFind = ""; }
#if ( $grepArg eq "" ) { $grepArg = " -n -I -i -C 3 "; }
if ( $grepArg eq "" ) { $grepArg = " -n -I -i "; }
if ( $output eq "" ) { $output = "~/tmp/find_$strings[0].rslt" }

# Start to build up the find argument

foreach ( @prunes ) { $arg .= " -not \\( -type d -name \"*$_*\" -prune \\) " }
foreach ( @ands ) { $arg .= " -name \"*$_*\" " }
foreach ( @nots ) { $arg .= " -not -name \"*$_*\" " }
if ( @fileNames > 1 ) { $arg .= " \\( "; }
foreach my $index (0..$#fileNames) {
if ( $index > 0 ) { $arg .= " -o "; }
$arg .= " -name \"*$fileNames[$index]*\" ";
}
if ( @fileNames > 1 ) { $arg .= " \\) "; }

# Create the comands and execute
foreach my $string ( @strings ) {
if ( $replaceWithSpace ne "" ) { $string =~ s/$replaceWithSpace/ /g; }
$cmd = "find @dirs $arg $endFind -follow ";
$cmd .= " -print0 | xargs -0 grep $grepArg '$string' $endGrep" unless $noGrep;
print " $cmd \n" unless $verbosity eq "quiet";
$findRslt = qx"$cmd" unless ( $dryRun );
if ( $findRslt =~ m/[a-z]/i ) {
$findRsltFound++;
push ( @outputList , $findRslt );
}
}

if ( ( $findRsltFound < 1 ) && ( ! $dryRun ) ) {.
die " !ERROR! ---> \"$strings[0]\" not found in dir(s) @dirs \n";.
}

# Generate my output
if ( $displayOnly ) {
print @outputList;
} else {
open OUT_FILE, "> $output" or die "cant open $output : $!";
print OUT_FILE @outputList;
close OUT_FILE;
#print @outputList;
unless ( $noVim ) {
my $cmd3;
my $gvimSearch = "";
# The search +/ does not work if using --remote in vim!!! SUCKS
$gvimSearch = " +\"/$strings[0]\" " unless ( $vimCmd =~ m/remote/ );
$cmd3 = "csh -c \"$vimCmd $output $gvimSearch \"";
print " $cmd3 \n";
system("$cmd3") == 0 or die "system $cmd3 failed: $?" unless ( $dryRun );
}
}


__END__

=head1 NAME

find.pl help

=head1 SYNOPSIS

find [options]
Options:
-help This help message
-dirs Directory to search*
-output Alternative output filename
-type Find's type (default = "-type f")
-endGrep|eg Additional grep arguments
-fileNames|fn File Name (default = "*")
-arg Additional find Arguments
-endFind|ef Additional find Arguments for the end of find command
-strings Strings grep will use to search*
-grepArg|ga Alternative grep arguments (default = " -n -I -i -C 3 ")
-nots Additional find logic*
-ands Additional find logic*
-prunes Trims directories from search
-displayOnly|do Does not generate a file
-fno Search for the file name only (no grep)
-noVim|nv Do not open output file with vim
-dryRun|dr Run the script but print the commands do not actually run them

Examples:

find.pl stringtofind
find.pl -fn filetofind stringtofind
find.pl -string stringtofind -fa "-maxdepth 1"
find.pl -s stringtofind -fa "-maxdepth 1"
find.pl stringtofind -fa "-maxdepth 1"
find.pl stringtofind -prune directorynottosearch
find.pl stringtofind -not log
find.pl string_with_spaces -replaceWithSpace _

=cut

Faster editting of linux script files

If you are like me you want to open your scripts quickly without having to find where they are and then opening it up. Use my little vich.pl script to automatiically find and open the script with one command:

##### FILE: vich.pl #####


#!/usr/local/bin/perl -w
use strict;
my $whichOutput = qx` which $ARGV[0]; `;
$whichOutput =~ s/\s+$//;
stat($whichOutput);
print "Readable\n" if -r _;
print "Writable\n" if -w _;
print "Executable\n" if -x _;
print "Setuid\n" if -u _;
print "Setgid\n" if -g _;
print "Sticky\n" if -k _;
print "Text\n" if -T _;
print "Binary\n" if -B _;
if ( ( $whichOutput =~ m/^\// ) && ( -T $whichOutput ) ) {
qx`gvim -geometry 100x50 $whichOutput `;
} else {
my $myoutput = "~/tmp/vich.out";
open OUT_FILE, "> $myoutput" or die "cant open $myoutput : $!";
print OUT_FILE $whichOutput;
close OUT_FILE;
qx`gvim -geometry 100x50 $myoutput `;
}
##########################################
# END OF FILE
##########################################



Make an alias:
> alias vich vich.pl

Now instead of doing:

> which myscript
/usr/bin/myscript
> vim /usr/bin/myscript

you can just do:

> vich myscript

Just a little thing to speed things up..

Make your config files very flexible (w/ Perl eval)

I have to make my config files flexible so that I can allow my users a lot of options and even override functions. I can do this by making my config file "eval"-able. Meaning that the config file is actually Perl code that can be interpreted by the calling Perl script.

Most of the code was taken from:

http://www.usenix.org/publications/perl/perl13.html

Here is my proof of concept code:

##### FILE: evalConfigExample.pl #####


#!/usr/bin/perl -w
# Filename: evalConfigFileExample.pl
# This is a proof of concept script to show how one can use a perl script as a config.
# file. This allows much more freedom when declaring a configuration file. One can
# call functions, redefine functions and generally do anything that you could.
# normally do in a regular perl script. Ex:
#; ~/bin/evalConfigFileExample.pl -file ~/bin/evalConfigFileExample.cfg

use Getopt::Long;
use Data::Dumper;
use Pod::Usage;

my $page_info ;
my %opts = ();
my $ok = GetOptions(\%opts,
'file=s',
'help'
);
if ( !$ok ) { die; }
pod2usage(1) if (exists ( $opts{help} ) );

sub printer {
print "In Perl prog\n";
}

sub hello_world {
print "Calling hello_world from eval'd config file!\n";
}

sub parse_cfg {
my($file) = @_;
delete($INC{$file});
eval('require("$file")');
die "*** Failed to eval() file $file:\n$@\n" if ($@);
print ("VENDOR = $VENDOR \n");
}

printer();
parse_cfg( $opts{file} );
print ("VENDOR = $VENDOR \n");
printer();

##########################################
# END OF FILE
##########################################



##### FILE: evalConfigExample.cfg #####


#evalConfigFileExample.cfg
$VENDOR = "Sun";
$HARDWARE = "Sparc";
$OS = "Solaris";
$VERSION = "2.5";
$HOSTNAME = `/bin/hostname`;
$PSCMD = "/usr/bin/ps -ef";
hello_world();

sub printer {
print "In required file\n";
}

##########################################
# END OF FILE
##########################################



The output should look like this:


> evalConfigFileExample.pl -file evalConfigFileExample.cfg
In Perl prog
Subroutine printer redefined at evalConfigFileExample.cfg line 9.
Calling hello_world from eval'd config file!
VENDOR = Sun
VENDOR = Sun
In required file