SpiderOAK Server Backup

I manage quite a few servers for various clients. Most of them are fairly small in size and backup is done via a bash script I wrote many years ago. For anyone who’s interested, I will publish the full script in a separate post soon.

The script assembles files, exports databases and crontabs and compresses everything into a single gzipped tar archive. It then encrypts the archive and uploads the encrypted data via ncftp to an offsite backup host.

This works great for a while but as the data grows so does the size of the backup.

As an aside to this if anyone is looking for a great FTP backup service I can thoroughly recommend ADrive as being very reliable and excellent value for money.

Simple Incremental Encrypted Backups

As they do, servers tend to grow and the one big file approach ends up with bigger and bigger backup archives that fail to upload more and more frequently.

So I started looking for a better way. For me it had to be 1) simple 2) incremental 3) encrypted and 4) (if possible) have a nice desktop client that I could access it through.

I’d recently started using SpiderOAK as a replacement for Dropbox due to privacy concerns. While searching for a simple server backup solution I came across their command line documentation.

So I thought I’d give it a go. Bingo! It was the server backup solution I’d been looking for.

So here it is my quick guide to setting up simple server backup using the SpiderOakONE command line utility on the Debian Linux operating system.

Installation

This guide assumes you already have a SpiderOakONE account configured and ready to go.

The client is easy to install, you just need to add in the SpiderOak repository, their gpg key and apt-get away:

echo 'deb http://apt.spideroak.com/debian/ stable non-free' >> /etc/apt/sources.list
gpg --keyserver pgpkeys.mit.edu --recv-key 573E3D1C51AE1B3D
gpg -a --export 573E3D1C51AE1B3D | sudo apt-key add -
apt-get update
apt-get install spideroakone

Now that you have the client installed, you need to connect it to your SpiderOak account. Enter the following and follow the on-screen prompts:

SpiderOakONE --setup=-

Once you’ve entered the requested information, it will start to synchronise with the SpiderOak servers. This can take a few minutes so probably a good time to get yourself a coffee!

Talking about coffee, if you find this or any of the other posts on my site helpful, perhaps you’d like to (shameless plug coming up) buy me one as well! There’s a link at the bottom of the post.

Running A Backup

Ok, so now that you have everything installed and ready to go, how do you run a backup. The simplest way is via the comamnd line, so for example:

SpiderOakONE --backup=/var/www

Would back up the contents of /var/www directory to SpiderOak’s encrypted backup servers.

Scheduling A Backup

You can take things a step further by creating a backup script and scheduling it as a cron job. You can also include database and crontab backups in this script, or anything else you need. The example bash script below might be useful as a starting point:

#!/bin/bash

# setup config
# where to put the db backups
BK_DEST="/root/data"
# mysql database backup login
BK_MYSQLSERVER="localhost"
BK_MYSQLUSER="<backup user>"
BK_MYSQLPASSWORD="<backup user password>"

# array of mysql databases to backup
BK_MYSQLDBS=( "db1" "db2" "etc" )
# if no databases 
# BK_MYSQLDBS=( )
echo "Starting backup"

# prepare for backup and create db backup folder if it doesn't already exist
if [ ! -d "$BK_DEST" ]; then
  # check that list of databases is not empty
  if [ -n "$BK_MYSQLDBS" ]; then
    echo " Creating local backup folder $BK_DEST"
    if ! mkdir -p $BK_DEST; then
      echo " PROBLEM CREATING BACKUP FOLDER"
      exit 1;
    fi
    echo " Creating backup folders"
    mkdir $BK_DEST/mysql
    mkdir $BK_DEST/crontabs
  fi
fi

# backup mysql databases listed in BK_MYSQLDBS
echo "Backing up databases"
for BK_DB in ${BK_MYSQLDBS[@]}; do
  echo " Processing $BK_DB"
  if ! eval "mysqldump -u$BK_MYSQLUSER -p$BK_MYSQLPASSWORD --triggers --routines --single-transaction --flush-logs $BK_DB |gzip -c > $BK_DEST/mysql/$BK_DB.sql.gz"; then
    echo " PROBLEM DUMPING DATABASE"
  fi
done

# spideroak
SpiderOakONE --backup=$BK_DEST
SpiderOakONE --backup=/var/www
SpiderOakONE --backup=/etc

You will notice that the MySQL database backup just overwrites the backup file each time. This is because the SpiderOak client takes care of the versioning and stores progressive incremental backups on their servers until you delete them.

Email Notifications

You can set up cron to email you the results of the backup. Nullmailer is my favourite and you can find more information that here.

The SpiderOak client also comes with all kinds of advanced backup commands including their own scheduling system but those are way beyond the scope of this article. Keeping it simple works for me, but you can find more by visiting the SpiderOak Command Line Reference documention.

Thanks for reading!

---