prompt:> mkdir /tmp/test
prompt:> cd !$

Now we'll create a few temporary html files to rename.

prompt:> for i in {1..10}; do touch $i.html; done

Now to rename them en masse:
First way (courtesy of Basketcase):

prompt:> for i in $(ls *html); do mv $i $(echo $i | sed 's/\(.*\)html/\1htm/'); done

Second way (courtesy of myself):

prompt:> for i in $(ls *html); do mv $i $(basename $i .html).htm; done

First way breakdown:
	1. loop through all the specified files, *html in our case.
	2. here is where the magic with sed happens, echo the filename $i, then use sed to greedy match our pattern and substitute it with the new pattern. We use the first part of the matching pattern with "\1" (which will expand to 1. 2. 3., etc) and then replace the last part of the pattern. As an example of this try in the working directory "for i in $(ls); do echo $i | sed 's/\(.*\)htm/\1/'; done" without doing the replacement, it should print all the files and the ones that match should be printed as "filename.", so in our dir it should look something like 1. 2. 3. 4. 5., etc.

Second way breakdown:
	1. loop through all the specified files, *html in our case.
	2. use basename utility to strip off the first pattern (.html) as we move the file to .htm. 

So I created 10k files 1-10000.html and timed the following two commands, this is by no means scientific, I was just curious what the timing was on each of them with more than 10 files.

On 10k files with first method:

prompt:> time for i in $(ls); do mv $i $(echo $i | sed 's/\(.*\).html/\1htm/'); done
 
real	1m36.604s
user	0m15.842s
sys	0m58.361s

On 10k files with second method:

prompt:> time for i in $(ls); do mv $i $(basename $i .htm).html; done
real	1m4.737s
user	0m9.260s
sys	0m38.631s

First way, with backtick `` interpolation:

prompt:> time for i in `ls *html`; do mv $i `echo $i | sed 's/\(.*\)html/\1htm/'`; done
 
real	1m18.076s
user	0m15.478s
sys	0m56.438s

Addendum:
Basketcase also pointed out that using perl rename is much faster, though since I am on an OSX box, it is not native and hence why I personally chose not to go that route.

linux prompt:> time rename  's/\.html/\.htm/' *.html
 
real    0m0.270s
user    0m0.204s
sys    0m0.064s

So I have finally decided after years of using imperative style languages that I am going to Learn me some Haskell for a greater good, as I have been curious about the functional side of programming. Now it has taken me quite some time to get here, I have been toying with the idea of functional programming for quite a while, just haven’t gotten around to it as the imperative languages seem to be just fine for what I do for a living, Systems Administration/Engineering. I have been toying with either learning Haskell or Erlang, and have decided on Haskell. Why?, well I don’t really know just kind of happened that I chose Haskell.

About a day after I started reading Learn you a Haskell for a Great Good! I noticed this article – (The Downfall of Imperative Programming) fly across my twitter stream from @LordCope. Now, it is a well written article and surprisingly the comments are for the most part well thought out (I really abhor reading commentary on the internets), and I have to agree with what Bartosz says here in response to a comment about functional programming being the downfall of the imperative languages, “Of course it’s an exaggeration :-) . Just like object oriented programming didn’t eliminate C, Haskell will not eliminate Java. It will just dislodge it into its well deserved niche.” I personally, and honestly think that to be the best that one can be it is sometimes necessary to open one’s mind and try new things, and for me right now that is Haskell and functional programming, and so far I am really enjoying myself.

After I finish with Learn you a Haskell for Great Good! I am going to move on to Real World Haskell which is also freely available online, and while I am not associated with any of the authors in any way, shape or form, if you like the books I strongly encourage you to purchase them as they require a lot of effort to write.

This does not stave off the infamous “rm -rf – pebkac error” but to keep myself from general stupidity I tend alias rm, cp and mv first and foremost (that way for the most part if I make a bad decision at least I was prompted to):

alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'

Also, back in the day, when I started messing with Linux, I loved the terminal colors, now not so much. So I alias away the colors, make them appear in columns and with characters to denote the type of file (‘/’ for directory, ‘*’ for executable, etc.).

alias ls='ls -CF'

Because “maths are teh hard” I tend to alias df so that it appears in human readable format. Sometimes I’ll alias free, but not always – sometimes I like to live dangerously. Conversely since I tend to work mainly on OSX and free isn’t a built in (yes I know it can be added with ports), I tend to just forget about it.

At the end I will then setup personal aliases that are meaningful to me, but may not have any meaning to someone not living inside my head. For instance if there are a few hosts that I know that I will be sshing to quite frequently, like say jump hosts, I will alias them down to singular commands, usually the hosts name to make getting there quicker and easier.

Note: this was done from my MBP, so in Linux ymmv*.

sed:

Before:
df -h | grep disk
 
/dev/disk0s2    465Gi  123Gi  342Gi    27%    /
/dev/disk1s2    110Mi   94Mi   16Mi    86%    /Volumes/VirtualBox
/dev/disk2s0s2   25Mi   22Mi  2.7Mi    90%    /Volumes/VZAccess Manager
 
After:
df -h | sed -n '/^Filesystem/p;/disk/p'
 
Filesystem       Size   Used  Avail Capacity  Mounted on
/dev/disk0s2    465Gi  123Gi  342Gi    27%    /
/dev/disk1s2    110Mi   94Mi   16Mi    86%    /Volumes/VirtualBox
/dev/disk2s0s2   25Mi   22Mi  2.7Mi    90%    /Volumes/VZAccess Manager

awk:

Before:
df -h | grep disk
 
/dev/disk0s2    465Gi  123Gi  342Gi    27%    /
/dev/disk1s2    110Mi   94Mi   16Mi    86%    /Volumes/VirtualBox
/dev/disk2s0s2   25Mi   22Mi  2.7Mi    90%    /Volumes/VZAccess Manager
 
After:
df -h | awk '/^Filesystem {print;}; /disk/ {print;}; {next;};'
 
Filesystem       Size   Used  Avail Capacity  Mounted on
/dev/disk0s2    465Gi  123Gi  342Gi    27%    /
/dev/disk1s2    110Mi   94Mi   16Mi    86%    /Volumes/VirtualBox
/dev/disk2s0s2   25Mi   22Mi  2.7Mi    90%    /Volumes/VZAccess Manager

sed breakdown:
With the sed line we are matching any line that begins with Filesystem “/^Filesystem” and printing it “/p”, then matching any line that has disk “/disk” and printing it “/p”. The “-n” option to sed tells it to only print what we are looking for – from the sed manpage on OSX – “-n By default, each line of input is echoed to the standard output after all of the commands have been applied to it. The -n option suppresses this behavior.” If you were to remove the “-n” switch it would look something like this:

 
df -h | sed '/^Filesystem/p;/disk/p'
 
Filesystem       Size   Used  Avail Capacity  Mounted on
Filesystem       Size   Used  Avail Capacity  Mounted on
/dev/disk0s2    465Gi  123Gi  342Gi    27%    /
/dev/disk0s2    465Gi  123Gi  342Gi    27%    /
devfs           199Ki  199Ki    0Bi   100%    /dev
map -hosts        0Bi    0Bi    0Bi   100%    /net
map auto_home     0Bi    0Bi    0Bi   100%    /home
/dev/disk1s2    110Mi   94Mi   16Mi    86%    /Volumes/VirtualBox
/dev/disk1s2    110Mi   94Mi   16Mi    86%    /Volumes/VirtualBox
/dev/disk2s0s2   25Mi   22Mi  2.7Mi    90%    /Volumes/VZAccess Manager
/dev/disk2s0s2   25Mi   22Mi  2.7Mi    90%    /Volumes/VZAccess Manager

printing the matches multiple times due to matching on the “/^Filesystem” and matching on the “/disk”.

awk breakdown:
With the awk one, we are matching the line that begins with “/^Filesystem” and printing it, then matching the line with “/disk/” and printing, then followed by a statement “{next;}” to keep processing and skip unless the pattern matches.

Also while writing the post, thought of another one that it might be useful, though it really depends on what you’re running on your system. Thought of a little netstat example though I will go with the caveat here that netstat can easily make this a moot point, take this with a grain of salt.

Before:
 
netstat -anl | grep -i listen
 
tcp46      0      0  *.9292                 *.*                    LISTEN     
tcp46      0      0  *.9302                 *.*                    LISTEN     
tcp46      0      0  *.9301                 *.*                    LISTEN     
tcp46      0      0  *.9200                 *.*                    LISTEN     
tcp46      0      0  *.9300                 *.*                    LISTEN     
tcp46      0      0  *.62056                *.*                    LISTEN     
tcp4       0      0  *.62056                *.*                    LISTEN     
tcp4       0      0  127.0.0.1.51093        *.*                    LISTEN     
tcp4       0      0  127.0.0.1.26164        *.*                    LISTEN     
tcp4       0      0  *.17500                *.*                    LISTEN     
tcp4       0      0  127.0.0.1.631          *.*                    LISTEN     
tcp6       0      0  ::1.631                                       *.*                                           LISTEN     
ffffff80147a27d0 stream      0      0 ffffff8017e13f00                0                0                0 /tmp/launch-H5oHbg/Listeners
 
After:
 
netstat -anl | sed -n '/^Proto/p;/LISTEN/p'
 
Proto Recv-Q Send-Q  Local Address          Foreign Address        (state)    
tcp46      0      0  *.9292                 *.*                    LISTEN     
tcp46      0      0  *.9302                 *.*                    LISTEN     
tcp46      0      0  *.9301                 *.*                    LISTEN     
tcp46      0      0  *.9200                 *.*                    LISTEN     
tcp46      0      0  *.9300                 *.*                    LISTEN     
tcp46      0      0  *.62056                *.*                    LISTEN     
tcp4       0      0  *.62056                *.*                    LISTEN     
tcp4       0      0  127.0.0.1.51093        *.*                    LISTEN     
tcp4       0      0  127.0.0.1.26164        *.*                    LISTEN     
tcp4       0      0  *.17500                *.*                    LISTEN     
tcp4       0      0  127.0.0.1.631          *.*                    LISTEN     
tcp6       0      0  ::1.631                                       *.*                                           LISTEN

Again, this makes the presumption that you know what you’re looking for, and if you tried to do that with ESTABLISHED in netstat it may be moot unless you narrowed down to less than a screen. Also I’ll leave the awk version and looking for specific ports / port ranges as an exercise for the reader. Also astute readers will notice that we lost the last output from the first netstat command due to not grepping indiscriminately.

I hope this information comes in handy, share and enjoy!

*For those that may not know, ymmv = Your Mileage May Vary.

http://code.google.com/edu/languages/google-python-class/index.html

http://i.imgur.com/YQafj.png

http://code.google.com/appengine/docs/python/gettingstarted/

http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-00-introduction-to-computer-science-and-programming-fall-2008/

http://diveintopython.org/toc/index.html

http://codingbat.com/

http://www.lightbird.net/py-by-example/

http://rgruet.free.fr/PQR26/PQR2.6.html

http://www.udemy.com/learn-python-the-hard-way/

http://learnpythonthehardway.org/

http://www.swaroopch.com/notes/Python

http://www.pythonchallenge.com/

http://www.youtube.com/user/thenewboston#p/search/0/4Mf0h3HphEA

Are they the “right” way to do them? They work! Probably not a “right” way to do them, as there is always more than one way to do something

Shell

#!/bin/bash
 
os=`uname`
 
if [ $os == 'Linux' ];
    then
    yesterday="$(date -d 'yesterday' +%m%d%y)"
    echo $yesterday
elif [ $os == 'SunOS' ];
    then
    yesterday=`TZ=GMT+24 date +%m%d%y`
    echo $yesterday
fi

Gnu Date

#!/usr/bin/bash
 
## Get Yesterday
yesterday="$(date -d 'yesterday' +%m%d%y)"

Powershell

$yesterday = (get-date -format mmddyy).adddays(-1)
$yesterday.tostring('MMddyy')

Python

 
import datetime
today = datetime.datetime.today()
one_day = datetime.timedelta(days=1)
yesterday = today - one_day
yesterday.strftime('%m%d%y')

Below is my first attempt. Watch for upcoming updated version.

import sys
 
if not sys.argv[1:]:
   mac = raw_input ("Enter MAC address\n")
   print "Mac is %s\n"; % (mac)
   #sys.exit(0)
else:
   mac = sys.argv[1]
   print "Mac is %s\n" % (mac)
 
##I'm checking to see if someone is giving me in the right format
maclen=len(mac)
if maclen == 12:
    ## converting to list, since strings are immuatable
    maclist=list(mac)
    ## defining positions on list
    positions=2,5,8,11,14
 
    for i in positions:
        maclist.insert(i,':')
    ##joining list at the ','
    newmac=",".join(maclist)
    ## stripping ',' out and converting to lowercase.  
    final12mac=newmac.replace(',','').lower()
 
    print final12mac
 
##assuming that 17 characters will either have - or : in MAC address    
elif maclen == 17:
    ##replacing - with : and converting to lowercase
    mac17final=mac.replace('-',':').lower()
    print mac17final
else:
    ## I'm telling them that if it isn't 12 or 17 characters, it has X chars
    print "There are only %s characters" % (maclen)