Incremental Backup

From btrfs Wiki
Revision as of 12:10, 30 November 2014 by Shriramana Sharma (Talk | contribs)

Jump to: navigation, search

BTRFS has some key capabilities which enable efficient backup solutions:

Instant, Atomic COW Snapshots
Since the snapshots are atomic, when a snapshot is restored it appears to applications as if a power loss had occurred (and the filesystem has gone back to an earlier state). Thus it is possible to backup databases without stopping them beforehand.
Incremental Snapshot Transfer
Efficiently determining and streaming the differences between two snapshots if they are either snapshots of the same underlying subvolume, or have a parent-child relationship. This is far quicker than e.g. rsync could, especially on large file systems. (For instance, rsync cannot be aware of mere metadata changes like filename, location etc but the FS itself is certainly aware of it.)

This page presents some approaches to leverage these capabilites.



Prerequisite is a fairly modern btrfs toolsuite, version 0.20rc1 seems to be good enough. Earlier versions may fail to list subvolumes when cleaner process is operating, and this could cause random failures during backup.

That said, many fixes have gone in btrfs send/receive and it is recommended to use at least a 3.14, or 3.15 kernel to have later patches. Similarly, you will want at least btrfs-tools 3.12, or later if available. Note that 32bit btrfs send will not work with a 64bit kernel as of 3.13.

Available Backup Tools

btrfs-subvolume-backup [1] by Marc MERLIN.
Script taking a snapshot and transferring it to another location via SSH. Keeps an configurable number of past snapshots
Backup tool with configurable snapshot retention (for example 1 per week for a month, one per month for a year), restore functionality, synchronization to and from other hosts
btrfs-sxbackup [3] by Marco.
Another more featureful backup script in python that lets you initiate backups from a server

Doing it by hand, step by step

When performing incremental backups, we will be working with two snapshots, one of them representing the time of the earlier backup, and the other representing the current backup. When the backup run completes, we can discard the earlier of the snapshots to prepare for the next incremental run.

Assuming that /home is the BTRFS volume you wish to backup, and backups are to be stored on another BTRFS volume called /backup and directory /backup/home, the procedure goes as follows.

Initial Bootstrapping

We will need to create a read-only snapshot of the volume that serves as the reference for the first backup. I will call this subvolume BACKUP. The subvolume is read-only because "btrfs send" requires read-only subvolumes to operate on. NB: there is currently an issue that the snapshots to be used with "btrfs send" must be physically on the disk, or you may receive a "stale NFS file handle" error. This is accomplished by "sync" after the snapshot:

   btrfs subvolume snapshot -r /home /home/BACKUP

Once created, we can distribute the initial copy into existing directory or subvolume /backup/home. The subvolume appears as /backup/home/BACKUP:

   btrfs send /home/BACKUP | btrfs receive /backup/home

Bootstrapping is now done. The subvolume /home/BACKUP is kept around to serve as local reference for the data that has been backed up, and it is needed for constructing the incremental backup for the next step.

Incremental Operation

During incremental backup, we make a new snapshot:

   btrfs subvolume snapshot -r /home /home/BACKUP-new

We can now send the difference between the old and new backup to the backup volume:

   btrfs send -p /home/BACKUP /home/BACKUP-new | btrfs receive /backup/home

Once this command completes, we should have these 4 subvolumes: /home/BACKUP, /home/BACKUP-new, /backup/home/BACKUP and /backup/home/BACKUP-new. We will now need to migrate the new backup as the old one, and do something for the old one. We could keep it around, maybe timestamped with the date of that backup, or just straight out delete it. Here, I am deleting it:

   btrfs subvolume delete /home/BACKUP
   mv /home/BACKUP-new /home/BACKUP
   btrfs subvolume delete /backup/home/BACKUP
   mv /backup/home/BACKUP-new /backup/home/BACKUP

But for instance, if you did want to keep a history of backups, perhaps you would snapshot one of the snapshot directories with something like:

   btrfs subvolume snapshot -r /backup/home/BACKUP /backup/home.$(date +%Y-%m-%d)

This concludes the incremental backup step.

Personal tools