Blog post: multi-device Btrfs

[joseivanlopez]

Specific blog post to summarize the multi-device Btrfs support by YaST.

PBI: https://trello.com/c/g5FtoTsm/1052-1-activate-probing-for-multidevice-btrfs-and-wrap-up-for-the-feature

[joseivanlopez]

Getting further with Btrfs in YaST

Btrfs is a modern file system for Linux aimed to implementing advanced features and focused on easy administration. Btrfs is becoming a very prominent technology in the Linux world, and it has been adopted as the default file system by SUSE Linux Enterprise and openSUSE Leap at least since versions 12 and 42 respectively.

Between the Btrfs major features we can find: subvolumes (separate internal file system roots), subvolumes-aware quota support, writable and read-only snapshots, efficient incremental backup and multiple devices support.

Most of these great Btrfs features are already supported by YaST, from the lower level at the libstorage-ng library, to the higher one, the YaST Expert Partitioner. But YaST Team does not stop working. And if you are an usual reader of this blog and you like following the YaST Team development progress, then most likely you already know that we have been adding support for multiple devices on Btrfs. And yes, we are already ready to present this awesome feature to our beloved users. So please, continue reading.

Multi-device Btrfs: welcome to YaST

Here you are. Multi-device Btrfs is now supported by YaST! But, what exactly is multi-device Btrfs about? It is quite simple: you can create a Btrfs file system over several devices! Think about RAID. Multi-device Btrfs is a quite similar idea. In fact, you can use it to completely replace software RAIDs.

But let's go with an example. Imagine you have two disks, /dev/sda and /dev/sdb, and you also have some partitions on the first disk. You can create a Btrfs file system over some devices at the same time, e.g., over /dev/sda2 and /dev/sdb, so you will have a configuration as the following:

        /dev/sda                /dev/sdb
            |                       |   
            |                       |   
     ---------------                |   
    |               |               |   
    |               |               |   
/dev/sda1       /dev/sda2           |   
                    |               |   
                    |               |   
                     ---------------
                            |   
                          Btrfs
                            |   
                            |   
                            @ (default subvolume)
                            |   
                            |   
                 -----------------------
                |       |       |       |   
                |       |       |       |   
              @/home  @/log   @/srv    ...

As you can see, in this example the multi-device Btrfs stands over two devices. Once you have the file system over several devices, you can configure it to do data stripping, mirroring, stripping + mirroring, etc. Basically everything what RAID does. In fact, you can configure how to treat the data and the Btrfs meta-data separately. For example, you could decide to do stripping with your data (by setting data RAID level to raid0 value) and to do mirroring with the Btrfs meta-data (setting it as raid1 level). For both, data and meta-data, you can use the following levels: single, dup, raid0, raid1, raid10, raid5 and raid6. This feature together to subvolumes makes Btrfs a very powerful tool. Basically, it allows you to manage your whole storage configuration from the file system itself. Usage of separate tools like Software RAID or LVM are simply dispensable when using Btrfs in all its glory.

And having said all that, it is a pleasure to announce that once again, YaST is here to make your life easier. YaST now supports to work with multi-device Btrfs file system in the Expert Partitioner and also in AutoYaST. But before going deeper into details, I would like to remark the effort of the YaST team regarding the storage stack refactoring that was accomplished for SLE 15 and openSUSE 15. Without that, including this kind of great features would be almost impossible. So, now let's go into details.

The Expert Partitioner and multi-device Btrfs

First of all, we adapted our library libstorage-ng to properly support multi-device Btrfs file systems. Once we had all its power, the next step was to redesign some parts of the Expert Partitioner UI to accommodate multi-device Btrfs. The first important change came to the Btrfs section:

As the image above shows, now the Btrfs section contains all Btrfs file systems, single- and multi-device ones. You can distinguish between them with a naked eyed by the name. The base name of the underlying device is used as keyword for a single-device Btrfs file systems (see "sda2" in the Btrfs section). For a multi-device Btrfs, a special name is used. Basically, it consists on the base name of its fist device closed between brackets and followed by dots (see "(sdb1 ...)" in our example). The Btrfs table contains the most relevant information about the file systems, and staying at this dialog, we have buttons to add, delete and modify each Btrfs file system.

In the previous screenshot you can see the dialog for adding a new Btrfs file system. Similar to RAID dialog, you have the available devices on the left and you can select the devices where you want to create the file system, and also you can indicate the data and meta-data RAID levels. Of course, the admissible RAID levels will depend on the number of selected devices. You will go to the second step of the Btrfs creation by clicking the "Next" button. In this second step, you can select the mount options and define the subvolumes, see the next image.

The "Modify" button offers two options: "Edit Btrfs" for editing the mount options and subvolumes, and "Change Used Devices" for modifying the devices used by the file system. Note that this second option can only be used when the file system does not exist on disk yet. Theoretically, Btrfs allows to add and remove devices from an already created file system, but a balancing operation would be needed after that. This balancing operation could take quite long, and for this reason it has been avoided in the Expert Partitioner.

Going to a specific Btrfs file system, we get a dialog with two tabs, see next screenshot:

Similar to the page for other devices, it contains an "Overview" tab with some details about the file system: mount point, file system label, UUID, data and meta-data RAID levels, etc. The file system can be also edited and deleted from this tab. Then, there is a second tab called "Used Devices" which contains a detailed list of the devices being used by the file system. Here, you have a button to edit that list:

And apart of all that, the Expert Partitioner has received quite more small improvements after including multi-device Btrfs file systems. For example, now the multi-device Btrfs file systems are considered first class citizens. And as consequence, the general list of devices now includes them, see the following image. Note that the "Type" column has also been improved to show more useful information:

And what else?

If you think all this is not enough for you, don't worry, we have more. For example, AutoYaST now adds support for multi-device Btrfs file systems. The official AutoYaST documentation will include a specific section about that. And of course, the storage proposal has learned to deal with your existing multi-device configuration when you are performing a new installation. The upgrade process is also ready to work with your multi-device Btrfs file systems.

And now it's your turn. We encourage you to test this new feature and report bugs if something does not work as you expected. And please, come with your ideas and further improvements and use cases. Stay tunned!

[joseivanlopez]

And after several improvements, the blog post is already published: https://lizards.opensuse.org/2019/06/19/getting-further-with-btrfs-in-yast/

Thanks @ancorgs !