Chapter 12. Backup Reference

Table of Contents

12.1. Configuration
12.2. Advanced Configuration
12.3. Listing Backup Contents
12.4. Restoring From Backup
12.5. Recovery From Earlier Backups
12.6. Offsite backup storage
12.7. Recovering from the offsite backup storage
12.8. Trimming the size of the local backups.
12.9. Making changes to the backup2l configuration

The Symbiosis system includes a component designed to handle backups, using the flexible backup2l software.

backup2l was selected due to its simplicity and flexibility, which allows it to be used easily. By default the backup software executes once per day and archives the contents of significant directories to a local directory. Before the actual backup takes place, the total space needed is calculated. If there is not sufficient storage space to accomodate the new backup, the backup operation will not proceed and no backups will be made. An error is generated in this case.

12.1. Configuration

In Symbiosis the Backup2l configuration is generated from the snippets in /etc/symbiosis/backup.d/conf.d/.

  • The local directories to backup (/etc/, /srv/, etc).
  • The destination to which the backups should be stored (/var/backups/localhost/)
  • The number of backups to keep.

12.2. Advanced Configuration

Additionally we’ve configured the backup software to easily execute a number of scripts before and after the backup is performed:

/etc/symbiosis/backup.d/pre-backup.d/
Any executable script located in this directory is executed, prior to a backup execution.
/etc/symbiosis/backup.d/post-backup.d/
Any executable script located in this directory is executed after a backup has completed.

12.3. Listing Backup Contents

To list the contents of your backup area you need to run backup2l with the "-l" flag:

all.1: /etc/.pwd.lock
all.1: /etc/GeoIP.conf.default
all.1: /etc/X11/Xresources/x11-common
all.1: /etc/X11/Xsession
all.1: /etc/X11/Xsession.d/20x11-common_process-args
all.1: /etc/X11/Xsession.d/30x11-common_xresources
all.1: /etc/X11/Xsession.d/40x11-common_xsessionrc
all.1: /etc/X11/Xsession.d/50x11-common_determine-startup
...

Here you will see the contents of the /etc/ directory which have been archived.

If you’d like to restrict this view you can apply a regular expression to filter the results. For example we can list the files which match the pattern passwd with this command:

~$ sudo backup2l -l passwd
Listing locations...
all.1: /etc/exim4/passwd.client
all.1: /etc/passwd
all.1: /etc/passwd-
all.1: /etc/phpmyadmin/htpasswd.setup
all.1: /etc/pure-ftpd/pureftpd.passwd
...

12.4. Restoring From Backup

To illustrate how this works, an example is used. We’re looking for a backup of the file /etc/passwd.

  1. First log in to your machine over SSH as admin.

  2. To find the available versions of the file, run sudo backup2l -l '/etc/passwd$'. The dollar sign is there to show that you want an exact match of /etc/passwd. The first time you run sudo you will be prompted for the admin password. The following output will be generated by backup2l.

    backup2l v1.5 by Gundolf Kiefer
    
    Active files in <all.1101>: 1
      found in all.1101:       0   (    1 left)
      found in all.11:         1   (    0 left)
    
    Listing locations...
    all.11: /etc/passwd

    This shows the latest available version of the file

  3. To recover it you should run sudo backup2l -r '/etc/passwd$'. The following output will be generated

    backup2l v1.5 by Gundolf Kiefer
    
    Active files in <all.1101>: 1
      found in all.1101:       0   (    1 left)
      found in all.11:         1   (    0 left)
    
    Restoring files...
      all.11.tar.gz: 1 file(s) using 'DRIVER_TAR_GZ'

That has restored the file to etc/passwd in the current directory. It is not recommended to run this program in the / directory, as any existing files will get overwritten.

12.5. Recovery From Earlier Backups

It is also possible to pick which version of a file you wish to restore.

  1. First login to your machine over SSH as admin

  2. Then, to show all available versions of a file, run sudo backup2l -a '/etc/passwd$'. Again, the first time you run sudo you will be prompted for a password. The following output is generated.

    backup2l v1.5 by Gundolf Kiefer
    
    Listing available files...
    all.101   -     1067 06/18/08 13:59:47 0000.0000 0644 /etc/passwd
    all.101   +     1118 06/19/08 11:29:10 0000.0000 0644 /etc/passwd
    all.108   -     1118 06/19/08 11:29:10 0000.0000 0644 /etc/passwd
    all.108   +     1153 08/27/08 10:25:45 0000.0000 0644 /etc/passwd
    all.11    -     1067 06/18/08 13:59:47 0000.0000 0644 /etc/passwd
    all.11    +     1153 08/27/08 10:25:45 0000.0000 0644 /etc/passwd
    all.1     +     1067 06/18/08 13:59:47 0000.0000 0644 /etc/passwd

    Note that the versions are not shown in date order, and that the dates are in the US mm/dd/yy format. In that list the + indicates that the file is new and thus contained in the archive file. A - indicates that the file has been removed (or replaced). Choose which backup you wish to recover from.

  3. To recover the file dated 19th June 2008, you need backup number 101 — remember the + indicates that it is present in that archive. To recover that file, run sudo backup2l -t 101 -r '/etc/passwd$'

    backup2l v1.5 by Gundolf Kiefer
    
    Active files in <all.101>: 1
      found in all.101:        1   (    0 left)
    
    Restoring files...
      all.101.tar.gz: 1 file(s) using 'DRIVER_TAR_GZ'

    Notice the -t 101 argument which specifies which backup we want to restore from.

We have now successfully restored the file to etc/passwd in the current directory. We can check by running ls -la etc/

total 16
drwxr-xr-x  2 root  root  4096 2008-09-09 09:56 .
drwxr-xr-x 14 root  root  4096 2008-09-09 09:51 ..
-rw-r--r--  1 root  root  1118 2008-06-19 11:29 passwd

12.6. Offsite backup storage

The Symbiosis system assumes that it has access to an associated external storage area. It will try and use rsync to upload the backups to this area, via a script named /etc/symbiosis/backup.d/post-backup.d/99-upload-backup.

If the host is on Bytemark’s network, this script can establish the backup space name automatically. Otherwise you can specify it manually by setting the full rsync path in /etc/symbiosis/dns.d/backup.name.

12.7. Recovering from the offsite backup storage

Before each backup a second script will synchronise the remote backup space locally, ensuring that a complete set of backups are held in both places. This means that if disaster strikes your machine, it is straightforward to recover your backups. This is done by running /etc/symbiosis/backup.d/pre-backup.d/00-download-backup.

This also helps to maintain the integrity of the differential backups provided by backup2l by replacing any files accidentally removed from the local backup directory before the backup starts.

12.8. Trimming the size of the local backups.

It is possible to reduce the size of the backups stored locally. The first thing to do is check the current status of the backups by running sudo backup2l -s. This will present a summary of the current backups. For example:

backup2l v1.5 by Gundolf Kiefer

Summary
=======

Backup       Date       Time  |  Size   | Skipped  Files+D |  New  Obs. | Err.
- ----------------------------------------------------------------------------
all.1        2010-08-10 02:52 |   41.7M |       0     3836 | 3836     0 |    0
all.11       2010-11-01 04:45 |   38.1M |       0     3935 | 1517  1418 |    0
all.12       2011-01-21 04:27 |   39.7M |       0     3985 |  561   511 |    0
all.121      2011-01-30 04:38 |   10.5M |       0     4001 |  137   121 |    0
all.122      2011-02-08 03:54 |    1.5M |       0     4029 |  129   101 |    0
all.123      2011-09-07 05:08 |   33.8M |       0     3892 | 1437  1574 |    0
all.124      2011-09-16 05:07 |    1.3M |       0     4791 |  956    57 |    0
all.125      2011-09-25 04:45 |    868K |       0     5676 |  928    43 |    0
all.126      2011-10-04 05:15 |   11.3M |       0     6559 |  990   107 |    0
all.127      2011-10-13 04:29 |    894K |       0     7444 |  928    43 |    0
all.128      2011-10-22 04:59 |    345K |       0     8329 |  935    50 |    0
all.13       2011-10-31 05:03 |   45.7M |       0     9218 | 6833  1600 |    0

Filesystem            Size  Used Avail Use% Mounted on
/dev/vda               10G  1.9G  7.6G  20% /

From here it is possible to see which levels of backups that can be pruned. In the above example the third-level backups all.121 to all.128 can be pruned, as there has been a subsequent second level backup, all.13. The downside of this is that any changes contained in those backups will be lost, and only changes from the all.12 will be available.

To prune these backups run sudo backup2l -p 121. This will then show Backup2l removing all.121 and all its dependent backups.

backup2l v1.5 by Gundolf Kiefer

Purging <121>...
  removing <all.121>
  removing <all.122>
  removing <all.123>
  removing <all.124>
  removing <all.125>
  removing <all.126>
  removing <all.127>
  removing <all.128>

Finally we need to make sure these deletions are synchronised to the remote backup space, to ensure that our deleted files do not mysteriously return again prior to the next backup run.

sudo /etc/symbiosis/backup.d/post-backup.d/99-upload-backup

Which will provide output similar to that shown below.

Sending backups to example.backup.bytemark.co.uk::example/example.default.bytemark.uk0.bigv.io...
building file list ... done
deleting localhost/all.lock
deleting localhost/all.128.tar.gz
....
deleting localhost/all.121.error.gz
deleting localhost/all.121.check
localhost/

sent 2.95K bytes  received 22 bytes  1.98K bytes/sec
total size is 400.59M  speedup is 134742.36

Those level three backups will no longer exist.

12.9. Making changes to the backup2l configuration

The configuration is kept in /etc/symbiosis/backup.d/conf.d. If you need to make changes, you should make them to the files in that directory and then run make to generate the live configuration file.