Tuesday, 7 August 2012

Raspberry Pi - Auto Backups

Anyway, I keep buggering my raspberry pi up, very annoying but inevitable when your experimenting, so I had to look at a way of backing up my Pi's SD card, so in the event of a failure I could restore from a stable copy.  Im also a lazy toe rag (although I prefer the term busy), so something as difficult as shutting down the pi, pulling the SD card out, putting it into a PC and taking a copy just wasn't going to work; I needed an automatted backup process that I didn't have to think about.

This turned out to be surprisingly easy providing your Pi is connected to a NAS drive or has a usb disk hanging off it.  If you want more info about connecting to a NAS, see this post.

I used the linux dd command to make an image of the SD card, which I then schedule to run using cron.

I took a backup of the SD card using dd referencing the physical disk directory /dev/mmcblk0 and a location on my NAS drive.  I doesn't have to be a NAS drive but, obviously, it does need to be location not on the SD card, so an external USB drive should work just as well.


sudo dd if=/dev/mmcblk0 of=/path/to/NAS/drive/raspberryPiSDCardBackup.img


Caution - be careful using the dd command, make sure you get the right syntax, particularily the if and of options, get them the wrong way round and your gonna need the backup you haven't taken!

Depending on the size of your SD card and speed of the external drive you are connecting too, this will take a little time, but when complete it should have created a .img file which is the same size as the total size of your sd card e.g. if you have an 8gb sd card you will end up with an 8gb image regardless of how much capacity is actually used.


I also stop services which run in the background (for me these are lighttpd and noip) before starting the backup and restarted them when finished to minimise the risk of taking a backup mid way through an update to the SD card.  As I also run other scheduled jobs using cron so I also suspend the cron service, so that other schedules jobs don't run while the backup is.

I created a script to do this called runSDCardBackup.sh


nano runSDCardBackup.sh


sudo /etc/init.d/lighttpd stop
sudo /etc/init.d/noip stop
sudo /etc/init.d/cron stop
sudo dd if=/dev/mmcblk0 of=/path/to/NAS/drive/raspberryPiSDCardBackup.img
sudo /etc/init.d/cron start
sudo /etc/init.d/lighttpd start
sudo /etc/init.d/noip start

Make the script executable:
sudo chmod +x runSDCardBackup.sh

I then scheduled this job using cron and that was it, it runs once a week on a monday morning at 2:30am.

Run crontab using the -e option for edit:

sudo crontab -e

Adding the following line to the bottom of the file, then using Ctrl X to save:

30 2 * * 1 /home/pi/runSDCardBackup.sh > /home/pi/SDCardBackup.log 2>&1

The first part "30 2 * * 1" is in the format "minute hour dayOfMonth Month dayOfWeek" tells cron to run when the minutes on the internal clock equal 30, the hour equals 2, for any day in the month, for any month when the day of the week is Monday.

The second part is the command which outputs the results to a log file.

For more information of how to use cron, see http://www.adminschoice.com/crontab-quick-reference.

17 comments:

  1. Looks cool, thanks for the info. I'd like to do something similar but have the img file be bootable so I could just write it back to the SD card (or another SD card) and have it boot from there. I think that wouldn't work with your method above? Or would it?

    ReplyDelete
  2. Fantastic, works great!! Now I can tweak the setup without worrying about losing all my settings.

    ReplyDelete
  3. Hi,

    I think that your script will restart cron etc. as soon as the dd command has been issued.
    The script does not wait for dd to complete before moving on to the next command.

    One way of making the script wait is to issue dd with a trailing '&' to make it a background process and then use wait $!
    wait $! means wait for the last background process to complete.

    dd if=/dev/mmcblk0 of=/path/to/NAS/drive/raspberryPiSDCardBackup.img &
    wait $!
    /etc/init.d/cron start

    You shouldn't need sudo on every line as you have scheduled the script to run using 'sudo crontab -e'
    This means the script will be run as root.

    If you scheduled the script to run by using 'crontab -e' it would be run as user and sudo would be required

    Regards
    anita2R

    ReplyDelete
    Replies
    1. dd blocks the following command until it has finished there is no need to run it in the background and wait for it to finish.

      I explicitly use sudo within the script as I don't always wait for the schedule to run to backup the sd card and like you say cron runs as whatever user set it up, so being explicit allows the script to be re-used in other setups.

      Delete
  4. Thanks for that information

    I had seen a number of posts about putting wait 'loops' into scripts to allow other commands to complete.
    I had tested wait$! on some other commands (not dd) using date stamps output to a file, so I knew it worked.

    It is clear from date stamping before and after dd, that the script does come to a halt and wait for it to complete.

    I have re-written my script - which was based on the one you posted - to remove the unnecessary wait command.

    I added to the script so that weekly backup names alternate - one week its an 'A' backup and the next week its a 'B' backup.

    Regards

    anita2R

    ReplyDelete
  5. Hi, great how-to,
    I was wondering if you could create a script to automatically compress the image .img after finished backup and delete .img file or if there is a way to backup not all the SD Size (I have 32GB one) but only space occupied by OS because I have just 6GB occupied by Raspbian.

    Thanks
    Steve

    ReplyDelete
  6. Does the cron run sudo without asking for the password? When I haven't used a sudo command in 5 minutes, it asks me to punch in the password again. How does your script handle this?

    ReplyDelete
    Replies
    1. If you setup cron using sudo (i.e. sudo crontab -e) all commands in that cron will run as root (and therefore not ask you for a root password)

      I suspect you are being asked for a password when you are using sudo because your user isn't in the /etc/sudoers file, see http://www.raspberrypi.org/phpBB3/viewtopic.php?f=28&t=17939. If you had your user to the sudoers file you wont be asked a password.

      Mart

      Delete
  7. As a newbie to Linux this looks great but how do I restore the image to the SSD card. Probably a dumb question but thought I woud put it out there as it might help someone else also.
    Cheers and thanks for sharing

    ReplyDelete
    Replies
    1. You'll dd the file back to the device. For instance:

      # dd if=/path/to/NAS/drive/raspberryPiSDCardBackup.img of=/dev/mmcblk0

      The only tricky part is that you'll likely need to do it from another linux box since the pi would presumably be hosed at that point. Once that's done, stick the sd card back into the pi and reboot.

      Delete
  8. Thank you for tutorial. I know this is quite old post, but I would highly recommend to use bs=512k parameter to speed up cloning of SD card.

    ReplyDelete
  9. When the cron job kicks off I get an error (see below)
    /bin/sh: 1: /home/pi/myNAS/myShare/runSDCardBackup.sh: Permission denied
    Any Thoughts?

    ReplyDelete
    Replies
    1. Did you setup cron job using sudo? Did you give the script execute permission?

      Delete
    2. Martian,
      Do you mean to add "sudo" before the script in the crontab?

      30 2 * * 1 /home/pi/runSDCardBackup.sh > /home/pi/SDCardBackup.log 2>&1

      or

      sudo 30 2 * * 1 /home/pi/runSDCardBackup.sh > /home/pi/SDCardBackup.log 2>&1

      Delete
    3. You could do that but if you setup the cron using sudo i.e.:

      sudo crontab -e

      the command will run as a super user anyway.

      Delete
  10. hi, does this method works while raspi is on running state ? or do i need to plugin my sdcard to another linux pc in order to backup ?

    ReplyDelete