Table of Contents
The Symbiosis system includes a component designed to handle backups, using the flexible backup2l software.
backup2l was selected due to its simplicity and flexibility, which allows it to be used easily. By default the backup software executes once per day and archives the contents of significant directories to a local directory. Before the actual backup takes place, the total space needed is calculated. If there is not sufficient storage space to accomodate the new backup, the backup operation will not proceed and no backups will be made. An error is generated in this case.
In Symbiosis the Backup2l configuration is generated from the
snippets in /etc/symbiosis/backup.d/conf.d/.
/etc/, /srv/, etc).
/var/backups/localhost/)
Additionally we’ve configured the backup software to easily execute a number of scripts before and after the backup is performed:
/etc/symbiosis/backup.d/pre-backup.d/
/etc/symbiosis/backup.d/post-backup.d/
To list the contents of your backup area you need to run backup2l with the "-l" flag:
all.1: /etc/.pwd.lock all.1: /etc/GeoIP.conf.default all.1: /etc/X11/Xresources/x11-common all.1: /etc/X11/Xsession all.1: /etc/X11/Xsession.d/20x11-common_process-args all.1: /etc/X11/Xsession.d/30x11-common_xresources all.1: /etc/X11/Xsession.d/40x11-common_xsessionrc all.1: /etc/X11/Xsession.d/50x11-common_determine-startup ...
Here you will see the contents of the /etc/ directory which have
been archived.
If you’d like to restrict this view you can apply a regular expression to filter the results. For example we can list the files which match the pattern passwd with this command:
~$ sudo backup2l -l passwd Listing locations... all.1: /etc/exim4/passwd.client all.1: /etc/passwd all.1: /etc/passwd- all.1: /etc/phpmyadmin/htpasswd.setup all.1: /etc/pure-ftpd/pureftpd.passwd ...
To illustrate how this works, an example is used. We’re looking for a
backup of the file /etc/passwd.
First log in to your machine over SSH as admin.
To find the available versions of the file, run sudo backup2l -l '/etc/passwd$'.
The dollar sign is there to show that you want an exact match of
/etc/passwd. The first time you run sudo you will be prompted
for the admin password. The following output will be generated
by backup2l.
backup2l v1.5 by Gundolf Kiefer Active files in <all.1101>: 1 found in all.1101: 0 ( 1 left) found in all.11: 1 ( 0 left) Listing locations... all.11: /etc/passwd
This shows the latest available version of the file
To recover it you should run sudo backup2l -r '/etc/passwd$'. The following output will be generated
backup2l v1.5 by Gundolf Kiefer Active files in <all.1101>: 1 found in all.1101: 0 ( 1 left) found in all.11: 1 ( 0 left) Restoring files... all.11.tar.gz: 1 file(s) using 'DRIVER_TAR_GZ'
That has restored the file to etc/passwd in the current directory.
It is not recommended to run this program in the /
directory, as any existing files will get overwritten.
It is also possible to pick which version of a file you wish to restore.
First login to your machine over SSH as admin
Then, to show all available versions of a file, run sudo backup2l -a '/etc/passwd$'. Again, the first time you run sudo you will be prompted for a password. The following output is generated.
backup2l v1.5 by Gundolf Kiefer Listing available files... all.101 - 1067 06/18/08 13:59:47 0000.0000 0644 /etc/passwd all.101 + 1118 06/19/08 11:29:10 0000.0000 0644 /etc/passwd all.108 - 1118 06/19/08 11:29:10 0000.0000 0644 /etc/passwd all.108 + 1153 08/27/08 10:25:45 0000.0000 0644 /etc/passwd all.11 - 1067 06/18/08 13:59:47 0000.0000 0644 /etc/passwd all.11 + 1153 08/27/08 10:25:45 0000.0000 0644 /etc/passwd all.1 + 1067 06/18/08 13:59:47 0000.0000 0644 /etc/passwd
Note that the versions are not shown in date order, and that the dates are in
the US mm/dd/yy format. In that list the +
indicates that the file is new and thus contained in the archive file.
A - indicates that the file has been removed
(or replaced). Choose which backup you wish to recover from.
To recover the file dated 19th June 2008, you need backup
number 101 — remember the + indicates that it is present in that
archive. To recover that file, run sudo backup2l -t 101 -r '/etc/passwd$'
backup2l v1.5 by Gundolf Kiefer Active files in <all.101>: 1 found in all.101: 1 ( 0 left) Restoring files... all.101.tar.gz: 1 file(s) using 'DRIVER_TAR_GZ'
Notice the -t 101 argument which specifies which backup we want to restore from.
We have now successfully restored the file to etc/passwd in the
current directory. We can check by running ls -la etc/
total 16 drwxr-xr-x 2 root root 4096 2008-09-09 09:56 . drwxr-xr-x 14 root root 4096 2008-09-09 09:51 .. -rw-r--r-- 1 root root 1118 2008-06-19 11:29 passwd
The Symbiosis system assumes that it has access to an associated external
storage area. It will try and use rsync to upload the backups to this
area, via a script named
/etc/symbiosis/backup.d/post-backup.d/99-upload-backup.
If the host is on Bytemark’s network, this script can establish the backup
space name automatically. Otherwise you can specify it manually by setting the
full rsync path in /etc/symbiosis/dns.d/backup.name.
Before each backup a second script will synchronise the remote backup space
locally, ensuring that a complete set of backups are held in both
places. This means that if disaster strikes your machine, it is
straightforward to recover your backups. This is done by running
/etc/symbiosis/backup.d/pre-backup.d/00-download-backup.
This also helps to maintain the integrity of the differential backups provided by backup2l by replacing any files accidentally removed from the local backup directory before the backup starts.
It is possible to reduce the size of the backups stored locally. The first thing to do is check the current status of the backups by running sudo backup2l -s. This will present a summary of the current backups. For example:
backup2l v1.5 by Gundolf Kiefer Summary ======= Backup Date Time | Size | Skipped Files+D | New Obs. | Err. - ---------------------------------------------------------------------------- all.1 2010-08-10 02:52 | 41.7M | 0 3836 | 3836 0 | 0 all.11 2010-11-01 04:45 | 38.1M | 0 3935 | 1517 1418 | 0 all.12 2011-01-21 04:27 | 39.7M | 0 3985 | 561 511 | 0 all.121 2011-01-30 04:38 | 10.5M | 0 4001 | 137 121 | 0 all.122 2011-02-08 03:54 | 1.5M | 0 4029 | 129 101 | 0 all.123 2011-09-07 05:08 | 33.8M | 0 3892 | 1437 1574 | 0 all.124 2011-09-16 05:07 | 1.3M | 0 4791 | 956 57 | 0 all.125 2011-09-25 04:45 | 868K | 0 5676 | 928 43 | 0 all.126 2011-10-04 05:15 | 11.3M | 0 6559 | 990 107 | 0 all.127 2011-10-13 04:29 | 894K | 0 7444 | 928 43 | 0 all.128 2011-10-22 04:59 | 345K | 0 8329 | 935 50 | 0 all.13 2011-10-31 05:03 | 45.7M | 0 9218 | 6833 1600 | 0 Filesystem Size Used Avail Use% Mounted on /dev/vda 10G 1.9G 7.6G 20% /
From here it is possible to see which levels of backups that can be pruned. In
the above example the third-level backups all.121 to all.128 can be pruned,
as there has been a subsequent second level backup, all.13. The downside of
this is that any changes contained in those backups will be lost, and only
changes from the all.12 will be available.
To prune these backups run sudo backup2l -p 121. This will then show
Backup2l removing all.121 and all its dependent backups.
backup2l v1.5 by Gundolf Kiefer Purging <121>... removing <all.121> removing <all.122> removing <all.123> removing <all.124> removing <all.125> removing <all.126> removing <all.127> removing <all.128>
Finally we need to make sure these deletions are synchronised to the remote backup space, to ensure that our deleted files do not mysteriously return again prior to the next backup run.
sudo /etc/symbiosis/backup.d/post-backup.d/99-upload-backup
Which will provide output similar to that shown below.
Sending backups to example.backup.bytemark.co.uk::example/example.default.bytemark.uk0.bigv.io... building file list ... done deleting localhost/all.lock deleting localhost/all.128.tar.gz .... deleting localhost/all.121.error.gz deleting localhost/all.121.check localhost/ sent 2.95K bytes received 22 bytes 1.98K bytes/sec total size is 400.59M speedup is 134742.36
Those level three backups will no longer exist.