My Backup Solution using BackInTime

Since a while now I'm trying to figure out the right way of doing backups on my Ubuntu Linux system. I have some requiremtens I want to descibe brefely before continuing.

Don't do regular full backups

I used sbackup so far but it regularly does a full backup, thus taking a long time (I have about 100GBs I want to be backuped) and using up a lot of disk space.

Store the backups on a remote system

Backup basics: keep your backup at a logicaly and spacialy seperate place from your computer. So I want it to work over network and please encrypt the file transfer. And do only send the data of changed files, keeping nettraffic low.

Don't bother me

Work in the background and just do your job. Don't tell me what you're doing unless I ask you to do so. I don't even want to know you exist. The only reason I interact with you is the worst case: I got to restore something.

Easy to setup and use

I want an easy to setup solution which doesn't need complicated settings and extensive care to be kept running. I don't want to change tapes or drives, free disk space tell you that the target is avilable or not, figure it out for yourself and do something sensible. And if I ever have to restore files I want an easy to use interface which enables me to quickly restore everything.

Restoring is also without the backup program easily possible

No, looking inside incremental .tar.gz files is not easy. More like complete snapshots I can browse easily and copy the files I want out of with any basic operating system. But keep the needed needed space low.

Try to backup every little change, ASAP

Most annoying with most backup solutions I think is the fact they only do a backup once a day or similar. With a full backup of 100GBs over the network this might take longer than the longest contiguous uptime of my computer while at home for the day. But I still want pretty much any changed file to be backed up, so I can undo very recent changes.

My solution

I really liked what I heard about TimeMachine from Apple. But I don't use Apple, so I was looking for something similar for Linux. There are a couple of promising projects like TimeVault and FlyBack, if they weren't deserted since about two years.

And there is BackInTime which works fine and is quite recently maintaned (June 2009). It even looks like it's going to be part of Ubuntu 9.10 Karmic Koala (release date Oct. 29th).

So what does BackInTime do that I like it so much?

1. You do one initial backup and then it does only incremental backups.


2. It creates snapshots of your files which you can easily browse without the backup program itself.
   


3. No space wasted through because files that didn't change just got linked in place by hardlinks instead of creating a full copy.


4. It's easy to install and setup. Well integrated into Gnome and KDE desktops.
   


5. After configuration it does not bother me again.


6. You can configure it to create a backup e.g. every 5 minutes.

But it does not store backups on remote systems. Well, you can mount a remote share and do the backup on that one. Luckily BackInTime internally uses rsync which uses Delta Encoding to reduce the amount of data to be sent over the network. But whenever the target is not available BackInTime gets annoying telling me about it every 30 seconds or so. Inacceptable. But there is something you can do about it.

BackInTime is called every 5 minutes by Cron, so I wrote a little wrapper script checking if the target computer is found in the local network and mounting the share prior to calling BackInTime.

#!/bin/sh
#
# Check for server, connect sshfs and start backintime
#

SERVER="192.168.1.1"
SHARE="/shares/backups/myPC"
MOUNT="/var/backups/backintime"
REMOTEUSER="backupuser"

# Is the server pingable?
# try three pings, wait maximum one sec for reply, be quiet
if ping -qc 3 -W 1 $SERVER > /dev/null; then

# check if the remote location is already mounted
# For this check to work the directory 'backintime' should exist in the
# mounted share (backintime creates it during it's first run)
if [ ! -d "${MOUNT}/backintime" ]; then
# make sure the user running this script is member of the "fuse" group,
# has write permission on the mountpoint
# and has a ssh key at the server so he can login without enrering a password
# and can mount this share manually
sshfs -o nonempty ${REMOTEUSER}@${SERVER}:${SHARE} ${MOUNT};
#echo "Verbindung aufgebaut";
fi

# now we have a connection, so let's start the regular backup program
nice -n 19 /usr/bin/backintime --backup-job
# unmount right after using the share
fusermount -uzq ${MOUNT};

else
# just in case there is no connection to the server we try to unmount the share
# be lazy (also unmount busy ressources) and be quiet
fusermount -uzq ${MOUNT};
fi

It's assumed your servers share is accessible via SSH and your user uses SSH keys to log in. There's a good howto on that on Ubuntus help pages.

Save it as e.g. ~/bin/backintimeprep.sh and change the crontab entry to something like

∗/5 ∗ ∗ ∗ ∗ ~/bin/backintimeprep.sh >/dev/null 2>&1

This script is very basic and fails if it finds another server with the same IP address.

Try it out and have fun! I just finished my initial backup and have it running in the background. Hopefully this ended my struggle for a good backup solution.

Using CIFS instead of SSHFS

My slug is very slow and I had a transfer rate of only about 1MB/s whilst totally hogging my CPU rendering it unusable for anything else during a backup. So I was looking for another solution with less CPU usage and better transfer rates.

CIFS is quick and doesn't use much CPU time. But it lacks the security of an encrypted connection which I decided to be negligible in my local network.

I assume you know your way around with samba i.e. how to add users. My /etc/samba/smb.conf on the server looks like this:

[global]
        workgroup = TREEHOUSE
        server string = %h server
        invalid users = root

[backups]
        comment = Backups
        path = /shares/backups/
        read only = No
        writeable = Yes
        write list = jan
        valid users = jan

In the script backintimeprep.sh change the lines for mounting and unmounting the sahres like this.

Mounting:

# you might need to fiddle around with the charset to make i.e. Umlauts work
# this assumes you have a file 'backup.smb' looking like this in your home directory:
#   username=backupuser
#   password=password
# Make sure you set the permissions so only you can read it
smbmount //${SERVER}/${SHARE} ${MOUNT} -o rw -o iocharset=utf8 -o credentials="backup.smb";

Unmounting:

unmount.cifs ${MOUNT};

Works great like this. Much better performance.

EDIT:

• added missing asterisks in crontab


• changed detection if the share is mounted


• unmount the share after using it


• added a section 'CIFS instead of SSHFS'


• changed cronjob to call backintimeprep.sh