Microsoft is sooo 1980s....

I mean really how come one of the oldest email clients around (outlook) does not even include instant messaging??? Yah know... I mean ... Really is it that difficult? I mean whateva... Microsoft rocks... NOT!

Google Sidewiki entry by Stephen

This is a really great company!

in reference to: http://proasix.com/ (view on Google Sidewiki)

How to colorize any text based output

I found this little tidbit the other day. Very useful for highlighting simulation output warnings and errors:

http://winterstorm.ca/hilite

Extract prices from craiglist

#!/usr/bin/perl
# This is a script that will help me search craigslist and extract
# price information and average the data to get what the going
# price of an item is
#
# Example:
#; ~/bin/cl_get_prices.pl -string=2006 -string=maxima
#; ~/bin/cl_get_prices.pl -string=2003 -string=maxima -cities=dallas
#; ~/bin/cl_get_prices.pl -string=2003 -string=maxima -verb=4
#; ~/bin/cl_get_prices.pl -string=ps3 -string=80gb -verb=4
#; ~/bin/cl_get_prices.pl -string=ps3 -string=80gb -trim=.333
#; ~/bin/cl_get_prices.pl -string=ps3 -string=60gb -trim=.2 -filter_on_title=1
#
# I think I am going to give three outputs. Mean, Median and Trimmed Mean.
# This way if there is outlier data it will be easy to spot in the Mean
# value.

use strict;
use Getopt::Long;
use List::Util qw(sum);
use Statistics::Descriptive;
use List::MoreUtils qw(uniq);

my @item_prices;
my $delay_between_searches = 5;

# Define the arguments
my %opts = ();
&usage unless &GetOptions(\%opts, 'help'
,'strings=s@'
,'cities=s@'
,'trim=s'
,'filter_on_title=s'
,'min=s'
,'max=s'
,'verbosity=i'
);
if ( @{opts{strings}} eq 0 ) { die " NO SEARCH STRING WAS GIVEN"; }
unless ( exists ${opts{cities}} ) {
push( @{$opts{cities}}, 'austin' );
}

$opts{trim} = .25 unless ( exists $opts{trim} );
$opts{min} = "" unless ( exists $opts{min} );
$opts{max} = "" unless ( exists $opts{max} );
$opts{verbosity} = 3 unless ( exists $opts{verbosity} );

foreach my $city ( @{$opts{cities}} ) {
&get_item_prices( \@item_prices, \@{$opts{strings}}, $city, $opts{min}, $opts{max}, $opts{filter_on_title} );
sleep( $delay_between_searches );
}

@item_prices = sort {$a <=> $b } @item_prices;
@item_prices = uniq @item_prices;
my $stat = Statistics::Descriptive::Full->new();
$stat->add_data(@item_prices);
$Statistics::Descriptive::Tolerance = 1e-10;
my $mean = $stat->mean();
my $var = $stat->variance();
my $tm = $stat->trimmed_mean($opts{trim});
my $std = $stat->standard_deviation();

print "Prices: @item_prices\n";
printf "Statistics: Mean: %10.4f Var: %10.4f Std Dev: %10.4f Trimmed mean: %10.4f\n", $mean, $var , $std , $tm ;
print "". scalar( @item_prices ) . " ITEMS WERE FOUND\n";
print "lowest:\t\t$item_prices[0]\n";
print "highest:\t$item_prices[$#item_prices]\n";
print "Mean:\t\t" . sum( @item_prices )/@item_prices ."\n";
print "Median:\t\t" . $item_prices[@item_prices/2] ."\n";
print "Trim Mean:\t" . $tm ."\n";

sub get_item_prices {
my ( $item_prices_ref, $strings_ref, $city, $min, $max, $filter_on_title ) = @_;
my $wget="http://$city.craigslist.org/search/sss?query=";
my $str_cat = "";
foreach my $str ( @$strings_ref ) {
$wget .= "$str_cat$str";
$str_cat = "+";
}
my $html = `wget -q -O - $wget`;
print $html . "\n" if $opts{verbosity} > 6;
my @chunks = split /(<\/?p>\s*)+/,$html;

# Filter out everything but the titles
my $num_chunks = scalar @chunks;
for ( my $idx = $num_chunks-1; $idx >= 0; $idx-- ) {
delete $chunks[$idx] unless $chunks[$idx] =~ m/^......\s-\s/;
print "$idx|$chunks[$idx]\n" if $opts{verbosity} > 4;
}

# Clean up the title and do additional filtering on the title
my $num_chunks = scalar @chunks;
for ( my $idx = $num_chunks-1; $idx >= 0; $idx-- ) {
my $rm_item = 0;
$chunks[$idx] =~ s/^.+html">//;
$chunks[$idx] =~ s/>//gs;
$chunks[$idx] =~ s/<//gs;
$chunks[$idx] =~ s/ / /gs;
$chunks[$idx] =~ s/<[^>]*>//gs;
if ( $filter_on_title eq "1" ) {
foreach my $str ( @$strings_ref ) {
if ( $chunks[$idx] !~ m/$str/i ) {
$rm_item = 1;
}
}
}
delete $chunks[$idx] unless $rm_item == 0;
}

# Extract price information from the titles
@chunks = sort @chunks;
foreach my $chunk (@chunks) {
if ( $chunk ne "" ) {
print " $chunk\n" if $opts{verbosity} > 3;
my @words = split(' ', $chunk);
foreach my $word (@words) {
if ( $word =~ m/^\$/) {
$word =~ s/(\$|,)//g ;
if ( $word =~ m/^\d+$/ ) {
push(@$item_prices_ref, $word);
}
}
}
}
}

}

HOTO: Create html "snapshot" from your gvim editor

Here is my .vimrc snippet that allows me to quickly save off a file in an HTML format with all of the syntax highlighting, etc..

The function is not working.. :( will have to work on it more later...

" Settings for :TOhtml
let html_number_lines=1
let html_use_css=1
let use_xhtml=1

""function not working
"function! MyToHtml()
"  normal :colorscheme default
"  normal :TOhtml
"  normal :colorscheme torte
"endfunction
"nmap <Leader>html :call MyToHtml()
nmap <Leader>html :TOhtml<CR>zR :w! ~/tmp/<C-R>%<CR> :q!<CR>
nmap <Leader>print :colorscheme default<CR> :TOhtml<CR>zR :w! ~/tmp/<C-R>%<CR> :q!<CR>zR:colorscheme torte<CR>

COLRM - Discovered a fun new tool!

It seems amazing to me that no matter how long I work in the world of Linux I am still to this day finding little tools to help my productivity. There are many different ways to remove certain columns of information from a text file but I just discovered colrm ( colrm first_col [second_col] ) which is a tool to do specifically that task. I am not sure how it does this behind the scenes so I am not sure how well it works if performance is critical but for a hackey little side thing it seems to do very well.

Here is an example where I removed columns 87 to the end and from column 1 to 69 leaving just column data from 70 to 86:

> cat ~/tmp/max_data.out  colrm 87  colrm 1 69 > ~/tmp/max_data2.out

A better unit conversion script (h2b b2h h2d d2h)

So I have been trying to get a robust unit conversion script for a while and I have had a few different solutions but I always had some troubles with them. bc is the calculator tool that comes with most linux distributions and is pretty widely available and is very good at doing the conversions but it too had some difficulties if the string containing the numbers was not formatted just right. So I use a small Perl wrapper script to do the formatting and then pass it on to bc. This seems to be working very well for me now. You can see the code below.

I also have some vim commands that call this script and allow me to do do conversions right from within vim (will add this vim code later). I also have some csh alias commands that I will add later as well.

It has proven to be very powerful and has improved my coding and debugging efficiency. Hope someone else finds it useful.


##### FILE: bc_conversion.pl ######

#!/usr/local/bin/perl -w
#   Use the bc function to do conversions..
#
##                 | check if called properly
#if ($#ARGV >= 0) {
if ($#ARGV < 2) {
    print "   Usage: $0    \n"              ;
    print "   used to wrap bc so you can set the base input and output \n"    ;
    print "   in order to do conversions. For example to convert binary \n"   ;
    print "   to hex, run the following command: \n"                          ;
    print "   bc_wrapper.sh 2 16 1001_1011 \n"                                ;
    print "   RESULT: 9B \n"                                                          ;
    exit
}

$ARGV[2] =~ s/_//g;
$ARGV[2] =~ tr/a-z/A-Z/;

# The following is a csh command that tells the tool bc the output base type,
# the input base type and the expression to run.
my $cmd = "bc << EOF \
obase=$ARGV[1];      \
ibase=$ARGV[0];      \
$ARGV[2];            \
quit;                \
EOF";

#print "  $cmd \n";
#system("$cmd") == 0 or die "system $cmd failed: $?";
my $result = `$cmd`;
# Make the result easier to read by inserting underscores
if ( ( $ARGV[1] eq "2" ) || ( $ARGV[1] eq "16" ) ) {
  $result = reverse $result;
  $result =~ s/(\w{4})/$1_/g;
  $result = reverse $result;
  $result =~ s/^_//g;
}
print "$result";