UNIX scripting tip: guard against bad directory names

Most UNIX/Linux folks write shell scripts, often in an ad hoc manner, and there's an easy mistake that can be very painful: attempting to copy/move to target directories that do not exist.

Let's look at a contrived example, where we want to move *.old to a backup directory only if there is a corresponding *.new:

for f in CL*.new
do
    oldname=`basename $f .new`.old          # create the .old name from .new

    if [ -f $oldname ]                      # does the file exist?
    then
        mv $oldname /home/backups
    fi
done

The loop control looks fine, and when I ran this last week on another machine it worked great - so what's the problem here?

The problem is that if the /home/backups directory is missing, it will move each source file to the single file /home/backups, effectively overwriting all but the last file. This destroys data. Ouch.

Ok, so let's add some error checking to be safe, creating the directory if it's not there:

[ -d /home/backups ] || mkdir /home/backup

for f in CL*.new
do
    oldname=`basename $f .new`.old

    if [ -f $oldname ]
    then
        mv $oldname /home/backups
    fi
done

Here, we've mistyped the directory name in the mkdir line, and the program destroys data the same way.

There are other ways to fix this, and at some point we have to take ownership of our own mistakes, but a really simple technique can mitigate much of this damage: always name directories with a trailing slash-dot:

for f in CL*.new
do
    oldname=`basename $f .new`.old

    if [ -f $oldname ]
    then
        mv $oldname /home/backups/.         # added here
    fi
done

The reference to slash-dot — no, not the geek website — insures that it's treated as a directory, and if it's not present (or if it's there but just a regular file), the command will fail outright:

$ ./myprog.sh
mv: cannot move `CL1.old' to `/home/backups/.': No such file or directory
mv: cannot move `CL3.old' to `/home/backups/.': No such file or directory

It may not be a graceful failure, but this little bit of defensive programming has saved my bacon a few times lately.