Conversion from Ext3

From btrfs Wiki
(Difference between revisions)
Jump to: navigation, search
m (superfluous category removed)
m (Btrfs Conversion from Ext3 or Ext4: tweak warning grammar)
(4 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
== Btrfs Conversion from Ext3 or Ext4==
 
== Btrfs Conversion from Ext3 or Ext4==
  
Btrfs has very few pieces of metadata that live in fixed locations, making
+
'''Warning: As of 4.0 kernels this feature is not often used or well tested anymore, and there have been some reports that the conversion doesn't work reliably. Feel free to try it out, but make sure you have backups'''.
it relatively easy to implement an in place conversion utility from other
+
filesystems. Copy on write algorithms allow Btrfs to preserve an unmodified
+
copy of the original FS, and allow administrator to undo the conversion,
+
even after making changes in the resulting Btrfs filesystem.
+
  
The conversion program [[Btrfs-convert|btrfs-convert]] uses libe2fs to read the Ext3/4 metadata, and
+
Btrfs has very few pieces of metadata that live in fixed locations, making it relatively easy to implement an in place conversion utility from other filesystems. Copy on write algorithms allow Btrfs to preserve an unmodified copy of the original FS, and allow administrator to undo the conversion, even after making changes in the resulting Btrfs filesystem.
uses the free blocks in the Ext3 filesystem to hold the new Btrfs. The basic
+
 
conversion algorithm works like this:
+
The conversion program [[Btrfs-convert|btrfs-convert]] uses libe2fs to read the Ext3/4 metadata, and uses the free blocks in the Ext3 filesystem to hold the new Btrfs. The basic conversion algorithm works like this:
  
 
* Duplicate the first 1MB of the device
 
* Duplicate the first 1MB of the device
Line 15: Line 11:
 
* Take references on Ext3 file data blocks from the Btrfs files
 
* Take references on Ext3 file data blocks from the Btrfs files
  
This does create copies of all the Ext3 metadata, but the Btrfs files
+
This does create copies of all the Ext3 metadata, but the Btrfs files simply point to the same blocks used by the Ext3 files. This shares the bulk of the blocks in use between the two filesystems. Because Btrfs uses copy on write for all file modifications, the original Ext3 version of the file data blocks is preserved.
simply point to the same blocks used by the Ext3 files. This shares the
+
bulk of the blocks in use between the two filesystems. Because Btrfs
+
uses copy on write for all file modifications, the original Ext3 version
+
of the file data blocks is preserved.
+
  
The first 1MB of the device is copied to an alternate location so that
+
The first 1MB of the device is copied to an alternate location so that Btrfs metadata may be written there. Undoing the conversion simply involves restoring these blocks.
Btrfs metadata may be written there. Undoing the conversion simply involves
+
restoring these blocks.
+
  
 
The result of the conversion looks something like this:
 
The result of the conversion looks something like this:
  
[[Image:Converter-description.png]]
+
[[File:Converter-description.png|[[Image:Converter-description.png|575px|Converter-description.png]]]]
 +
 
 +
The conversion program creates a snapshot in Btrfs that references all of the blocks used by Ext3, allowing the administrator to control how long the original Ext3 filesystem is preserved. If the admin wishes to recover the space used by Ext3, the snapshot can be deleted and the conversion will be permanent.
 +
 
 +
Until the snapshot is deleted, only blocks that were marked as free in Ext3 are used to hold new Btrfs modifications. This means the conversion can be undone at any time, restoring the original Ext3 FS. The converter creates a sparse file to hold all of the references to the original Ext3 filesystem, and the offsets in this file correspond to offsets on the block device. This allows the admin to mount the image file via readonly loopback and see the original Ext3 filesystem and the converted Btrfs filesystem at the same time.
 +
 
 +
==Example usage==
 +
 
 +
# Always run fsck first
 +
fsck.ext3 -f /dev/xxx
 +
 +
# Convert from Ext3/4->Btrfs
 +
btrfs-convert /dev/xxx
 +
 +
# Mount the resulting Btrfs filesystem
 +
mount -t btrfs /dev/xxx /btrfs
 +
 +
# Mount the ext3/4 snapshot
 +
mount -t btrfs -o subvol=ext2_saved /dev/xxx /ext2_saved
 +
 +
# Loopback mount the image file
 +
mount -t ext3 -o loop,ro /ext2_saved/image /ext3
  
The conversion program creates a snapshot in Btrfs that references all of
+
In the example above, the files in /ext3 and in /btrfs will be identical. Any modifications made in /btrfs will not show up in /ext3. If you wish to roll back the conversion:
the blocks used by Ext3, allowing the administrator to control how long the
+
original Ext3 filesystem is preserved. If the admin wishes to recover the
+
space used by Ext3, the snapshot can be deleted and the conversion will be
+
permanent.
+
  
Until the snapshot is deleted, only blocks that were marked as free in Ext3
+
# Completely unmount the Btrfs filesystem
are used to hold new Btrfs modifications. This means the conversion can be
+
  umount /ext3
undone at any time, restoring the original Ext3 FS. The converter creates a
+
umount /ext2_saved
sparse file to hold all of the references to the original Ext3 filesystem, and
+
umount /btrfs
the offsets in this file correspond to offsets on the block device. This
+
allows the admin to mount the image file via readonly loopback and see the
+
# roll back the conversion
original Ext3 filesystem and the converted Btrfs filesystem at the same
+
  btrfs-convert -r /dev/xxx
time.
+
 +
# use the original filesystem
 +
  mount -t ext3 /dev/xxx /ext3
  
Example Usage:
+
Or, if you wish to recover the space used by the original Ext3/4 FS and make the conversion permanent, simply delete /ext2_saved/image. Once the image is deleted, the disk will look something like this:
<pre>
+
# Always run fsck first
+
fsck.ext3 -f /dev/xxx
+
  
# Convert from Ext3/4->Btrfs
+
[[File:Converter-post.png|[[Image:Converter-post.png|536px|Converter-post.png]]]]
btrfs-convert /dev/xxx
+
  
# Mount the resulting Btrfs filesystem
+
== Before first use ==
mount -t btrfs /dev/xxx /btrfs
+
  
# Mount the ext3/4 snapshot
+
Suppose you're satisfied now and do not intend to rollback. Delete the saved subvolume:
mount -t btrfs -o subvol=ext2_saved /dev/xxx /ext2_saved
+
  
# Loopback mount the image file
+
# on a mounted filesystem
mount -t ext3 -o loop,ro /ext2_saved/image /ext3
+
btrfs subvol delete ext2_saved
</pre>
+
  
In the example above, the files in /ext3 and in /btrfs will be identical.
+
Wait until all it's fully deleted, this happens on the background. You can check whether there's a ''DELETED'' subvolume in the listing of ''btrfs subvol list -d /mnt''. Now any logical relation to the original filesystem is removed.
Any modifications made in /btrfs will not show up in /ext3. If you wish
+
to roll back the conversion:
+
  
<pre>
+
The new filesystem inherits the block placement and file data fragmentation. Due to this it is highly recommended to do full defragmentation and full rebalance before "production" use. It is not required for general operation but will impact performance. A notable caveat is that a balance can fail with "ENOSPC" if the defragment is skipped. This is usually due to large extents on ext being larger than the maximum size btrfs normally operates with (1 GB). A defrag of all large files will avoid this:
# Completely unmount the Btrfs filesystem
+
umount /ext3
+
umount /ext2_saved
+
umount /btrfs
+
  
# roll back the conversion
+
btrfs fi defrag -r /mnt
btrfs-convert -r /dev/xxx
+
btrfs balance start /mnt
  
# use the original filesystem
+
Alternative to defragment only files larger than 1 GB:
mount -t ext3 /dev/xxx /ext3
+
</pre>
+
  
Or, if you wish to recover the space used by the original Ext3/4 FS and make
+
find /mnt -type f -size +1G -print -exec btrfs fi defrag {} \;
the conversion permanent, simply delete /ext2_saved/image.  Once the image is
+
btrfs balance start /mnt
deleted, the disk will look something like this:
+
  
[[Image:Converter-post.png]]
+
Please note, that the balance process can take very long, you may want to do it in smaller steps, eg. metadata only, then data. See [[Balance Filters]].
  
[[Category:Documentation]]
+
[[Category:UserDoc]]

Revision as of 17:02, 11 June 2016

Btrfs Conversion from Ext3 or Ext4

Warning: As of 4.0 kernels this feature is not often used or well tested anymore, and there have been some reports that the conversion doesn't work reliably. Feel free to try it out, but make sure you have backups.

Btrfs has very few pieces of metadata that live in fixed locations, making it relatively easy to implement an in place conversion utility from other filesystems. Copy on write algorithms allow Btrfs to preserve an unmodified copy of the original FS, and allow administrator to undo the conversion, even after making changes in the resulting Btrfs filesystem.

The conversion program btrfs-convert uses libe2fs to read the Ext3/4 metadata, and uses the free blocks in the Ext3 filesystem to hold the new Btrfs. The basic conversion algorithm works like this:

  • Duplicate the first 1MB of the device
  • Duplicate directories and inodes, creating copies in Btrfs
  • Take references on Ext3 file data blocks from the Btrfs files

This does create copies of all the Ext3 metadata, but the Btrfs files simply point to the same blocks used by the Ext3 files. This shares the bulk of the blocks in use between the two filesystems. Because Btrfs uses copy on write for all file modifications, the original Ext3 version of the file data blocks is preserved.

The first 1MB of the device is copied to an alternate location so that Btrfs metadata may be written there. Undoing the conversion simply involves restoring these blocks.

The result of the conversion looks something like this:

The conversion program creates a snapshot in Btrfs that references all of the blocks used by Ext3, allowing the administrator to control how long the original Ext3 filesystem is preserved. If the admin wishes to recover the space used by Ext3, the snapshot can be deleted and the conversion will be permanent.

Until the snapshot is deleted, only blocks that were marked as free in Ext3 are used to hold new Btrfs modifications. This means the conversion can be undone at any time, restoring the original Ext3 FS. The converter creates a sparse file to hold all of the references to the original Ext3 filesystem, and the offsets in this file correspond to offsets on the block device. This allows the admin to mount the image file via readonly loopback and see the original Ext3 filesystem and the converted Btrfs filesystem at the same time.

Example usage

# Always run fsck first
fsck.ext3 -f /dev/xxx

# Convert from Ext3/4->Btrfs
btrfs-convert /dev/xxx

# Mount the resulting Btrfs filesystem
mount -t btrfs /dev/xxx /btrfs

# Mount the ext3/4 snapshot
mount -t btrfs -o subvol=ext2_saved /dev/xxx /ext2_saved

# Loopback mount the image file
mount -t ext3 -o loop,ro /ext2_saved/image /ext3

In the example above, the files in /ext3 and in /btrfs will be identical. Any modifications made in /btrfs will not show up in /ext3. If you wish to roll back the conversion:

# Completely unmount the Btrfs filesystem
umount /ext3
umount /ext2_saved
umount /btrfs

# roll back the conversion
btrfs-convert -r /dev/xxx

# use the original filesystem
mount -t ext3 /dev/xxx /ext3

Or, if you wish to recover the space used by the original Ext3/4 FS and make the conversion permanent, simply delete /ext2_saved/image. Once the image is deleted, the disk will look something like this:

Before first use

Suppose you're satisfied now and do not intend to rollback. Delete the saved subvolume:

# on a mounted filesystem
btrfs subvol delete ext2_saved

Wait until all it's fully deleted, this happens on the background. You can check whether there's a DELETED subvolume in the listing of btrfs subvol list -d /mnt. Now any logical relation to the original filesystem is removed.

The new filesystem inherits the block placement and file data fragmentation. Due to this it is highly recommended to do full defragmentation and full rebalance before "production" use. It is not required for general operation but will impact performance. A notable caveat is that a balance can fail with "ENOSPC" if the defragment is skipped. This is usually due to large extents on ext being larger than the maximum size btrfs normally operates with (1 GB). A defrag of all large files will avoid this:

btrfs fi defrag -r /mnt
btrfs balance start /mnt

Alternative to defragment only files larger than 1 GB:

find /mnt -type f -size +1G -print -exec btrfs fi defrag {} \;
btrfs balance start /mnt

Please note, that the balance process can take very long, you may want to do it in smaller steps, eg. metadata only, then data. See Balance Filters.

Personal tools