rsync Home Backup Script

I recently had trouble with a very good backup utility, rdiff-backup. I have always liked its functionality and it usually worked very well for me. However, I ran in to some trouble and somehow my backup information got corrupted, which kept the program from completing subsequent backups. This is not good. Because it was a bit of an obscure program and because I realized that I was using a program that I couldn't very easily dissect and because I was afraid that my actual data may become corrupt I decided to switch to another backup utility.

The process of searching for another backup utility turned south when I realized that there wasn't really any that did what I wanted. Here is all I wanted:

• Backup my /home/jkag/ directory, except all of the hidden files (.*)
• Keep incremental changes in an easy to get to location so that I could hand restore them as needed
• Be executable via cron
• Something written in Python would have been nice, but really, whatever worked would have been fine
  
• Keep a simple and easy to read log of everything that happened during the backup run

To me, this sounded like a pretty basic set of requirements for running backups. In fact, I figured everyone else (who is running Linux) would be doing the same thing. Yet, I was wrong, I found it very difficult to find a program that worked to my liking. I did look at http://www.diffingo.com/oss/fwbackups but that turned in to something much more complicated than I desired (installation in particular).

All of this history to say that I decided it would be easier to write my own backup script. I had done this several times in Windows, using xcopy. I knew that it would be fairly simple to write a bash script, utilizing rsync, to do all of this. So that's what I did, here it is!

#! /bin/bash

#Variables
BACKUPTIME="`date +%m-%d-%Y`"
SOURCEDIR="/home/jkag/"
BACKUPDIR="/opt/Backup/jkag"
LOGDIR="/opt/Backup/rsync-backup.log"
INCREMENTSDIR="/opt/Backup/jkag-increments/$BACKUPTIME"
MYTITLE="================================="
MYSUBTITLE="------------------"

#Log formatting
echo $MYTITLE >> $LOGDIR
echo $BACKUPTIME >> $LOGDIR
echo $MYTITLE >> $LOGDIR
echo "BEGIN BACKUP" >> $LOGDIR
echo $MYSUBTITLE >> $LOGDIR
echo >> $LOGDIR

#This script syncronizes the home directory with the backup directory and puts backups of changed and deleted files into a folder with the current date in the jkag-increments directory
mkdir $INCREMENTSDIR
rsync -avb --delete --exclude ".*" --backup-dir=$INCREMENTSDIR $SOURCEDIR $BACKUPDIR >> $LOGDIR

#Log formatting
echo >> $LOGDIR
echo "END BACKUP" >> $LOGDIR
echo $MYSUBTITLE >> $LOGDIR
echo >> $LOGDIR
echo >> $LOGDIR

However, the whole time I knew that I would really rather be writing this script in Python. So, after I wrote it out and tested it in bash I went ahead and rewrote it in Python. This was a lot of fun! Here are the Python scripts.

rsync-backup.py

#!/usr/bin/env python

#This script syncronizes the home directory with the backup directory and puts backups of changed and deleted files into a folder with the current date in the jkag-increments directory
import os
from time import asctime
import restformat

#Variables
backupruntime = asctime()
sourcedir = "/home/jkag/"
backupdir = "/opt/Backup/jkag"
incrementsdir = '/opt/Backup/jkag-increments/' + backupruntime

#Logging
backuplog = open('/opt/Backup/rsync-backup.py.log', 'a')
print >> backuplog, (restformat.title(backupruntime))
print >> backuplog, ('Making increments directory: "' + incrementsdir + '"\n')

#Make incrementsdir
os.mkdir(incrementsdir)

#Logging
print >> backuplog, (restformat.subtitle('BEGIN BACKUP'))
backuplog.close()

#Run rsync backup command with Linux logging
os.system('rsync -avb --delete --exclude ".*" --backup-dir=' + '"' + incrementsdir + '" ' + sourcedir + ' ' + backupdir + ' >> ' + '/opt/Backup/rsync-backup.py.log')

#Logging
backuplog = open('/opt/Backup/rsync-backup.py.log', 'a')
print >> backuplog, (restformat.subtitle('END BACKUP'))
print >> backuplog, ('\n\n')
backuplog.close()

restformat.py

#!/usr/bin/env python

def title(mytitle='reStructeredText Title'):
'Returns mytitle formatted as a reStructeredText title.'
return '=================================' + '\n' + mytitle + '\n' + '================================='

def subtitle(mysubtitle='reStructeredText Subtitle'):
'Returns mysubtitle formatted as a reStructeredText subtitle.'
return mysubtitle + '\n' + '---------------------------------'

Remember, one of my requirements was to have an easy to read log of what was going on, that is why I used reStructuredText formatting. Below is a sample log output from the Python script.

=================================
Mon May 11 21:46:27 2009
=================================
Making increments directory: "/opt/Backup/jkag-increments/Mon May 11 21:46:27 2009"

BEGIN BACKUP
---------------------------------
sending incremental file list
./
Unsaved Document 1
Desktop/
Desktop/computer.png
...[output truncated]...

sent 1279685 bytes received 6677 bytes 367532.00 bytes/sec
total size is 115368531000 speedup is 89685.90
END BACKUP
---------------------------------

Enjoy!