Issue #1241: Use BTRFS subvolumes for shared folders on BTRFS file systems.

[votdev]

Snapshots can be created per shared folder. The snapshot management UI allows to create and delete those snapshots. A restore feature has not been implemented by intention because those shared folders need to be unreferenced from every service beforehand. Instead shared folders should be created from these snapshots, which is a less error-prone approach.

• Subvolumes will be used for shared folders on BTRFS volumes.
• Add snapshot management for shared folders.
• A restore/rollback of snapshots has NOT been implemented by intention because it is difficult to do that when running services access the shared folder. Forcing the users to remove all shares beforehand (as suggested in the docs of a commencial NAS) does not make sense. Instead the user can easily create a shared folder pointing to the snapshot.
• Snapshots are NOT stored below the shared folder, instead they are stored in the root of the BTRFS filesystem in the .snapshots subvolume. This is done to do not pollute the shared folder which might cause side-effect, e.g. when syncing data with tools that do not know that .snapshots should be ignored.
• Snapshots are named like the shared folder and a timestamp in ISO 8601 format.

|- /srv/xxxx (Mount point BTRFS filesystem)
  |- .snapshots
  |- Shared folder 1
  |- Shared folder 2

Fixes: #312
Fixes: #1241

Signed-off-by: Volker Theile votdev@gmx.de

• References issue
• Includes tests for new functionality or reproducer for bug

[PrplHaz4]

This is a great feature and will really allow people to start using the best parts of BTRFS without having to know every detail of the implementation.

There seems to be a lot of churn in the community over snapshot naming being incompatible between some of the popular tools (btrbk, snapper), so it may be worth considering how those will interact.

[neheb]

Just tested this. Works great.

[paulsijben]

Thank you for adding this! I have my setup running nicely with btrfs.

One question, how do I automatically prune (too) old snapshots?

[wokkieman]

related to the question above, can I just remove the snapshot via command line or will this break somewhere else in the system?

[ryecoaaron]

@rjwonder yes

[wokkieman]

When having a shared folder on the root of my drive /srv/dev-disk-by-uuid-xxx/ I can snapshot the full thing. 'btrfs subvolume list ' is showing only .snapshots as subvolume. This works well for me as I like to feed the snapshot to snapraid (https://github.com/automorphism88/snapraid-btrfs).

When there used to be shared folders (now deleted), the sub volumes seems to stay. I think preventing me from snapping the full drive. When trying to setup a new share on /srv/dev-disk-by-uuid-xxx/ the snapshot creates an entry in .snapshots and all folders, but no files. I guess this has to do with BTRFS and the facts that snapshots never go into child sub volumes?

Would this be the best way to do full disk snapshots? Use CoW to copy the data to a different folder. Use delete subvolume on original (includes data deletion) and create a shared folder on /srv/dev-disk-by-uuid-xxx/? If I want shared folders on specific folders (e.g. for samba) then put them on the pool which is not BTRFS for OMV?
Update: this worked. Obviously snapshots on mergerfs merged pool are not possible

[votdev]

related to the question above, can I just remove the snapshot via command line or will this break somewhere else in the system?

Yes, OMV is not doing any magic here.

[votdev]

So the creation of every 'Shared Folder' on a BTFRS drive will create a "Subvolume" and therefore be ignored by a Snapraid sync...

Snapraid is no official plugin of OMV, so i only can enhance the documentation.

[votdev]

So the creation of every 'Shared Folder' on a BTFRS drive will create a "Subvolume" and therefore be ignored by a Snapraid sync...

There is a workaround for that situation. If the directory already exists, the shared folder creation mechanism does not create a new subvolume when you point to that directory in the shared folder creation dialog in the UI. The UI gives you more or less already that hint. This way you can create shared folders with classic directories on BTRFS file systems.

[ryecoaaron]

As I posted on the forum, until snapraid supports btrfs subvolumes, there is nothing I can do with the snapraid plugin to help.

[votdev]

As I posted on the forum, until snapraid supports btrfs subvolumes, there is nothing I can do with the snapraid plugin to help.

It's on snapraid to support subvolumes. Hey, these are regular directories from the file system consumer point of view. Everyone can access and process them as normal. So why ignoring them?

[MarcS1975]

Subvolumes are not filesystems. They actually just act like filesystems but are on a higher level of abstraction. Btrfs wiki "A Btrfs subvolume can be thought of as a separate POSIX file namespace, mountable separately by passing subvol or subvolid options to the mount(8) utility." Snapraid ignores Snapshots because of data integrity issues and my guess is SR will never support it. But this would go to far for this discussion here. Bottom line you decide to go down the Subvolume route it excludes Snapraid use. But thats your free choice of course.

…

On Sat, 3 Jun 2023, 09:53 Volker Theile, ***@***.***> wrote: As I posted on the forum, until snapraid supports btrfs subvolumes, there is nothing I can do with the snapraid plugin to help. It's on snapraid to support subvolumes. Hey, these are regular directories from the file system consumer point of view. Everyone can access and process them as normal. So why ignoring them? — Reply to this email directly, view it on GitHub <#1480 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMNNHGT4XSFNSH6UIYQ2KETXJL3QVANCNFSM6AAAAAAUPI5XBM> . You are receiving this because you commented.Message ID: ***@***.***>

[MarcS1975]

I commend your enthusiasm to focus on one filesystem and add more Plugin functionality for OMV, but what I used to love about OMV was always the great flexibility to freely choose filesystems and RAID management solutions. Now it all seems to go into BTRFS support as a filesystem, and especially as a RAID solution.

I assume you are aware of the endless issues of BTRFS. All the unfinished stuff, the severe RAID5/6 issues and the messy implementation of BTRFS storage namespaces.

https://arstechnica.com/gadgets/2021/09/examining-btrfs-linuxs-perpetually-half-finished-filesystem/

The btrfs-raid5 and btrfs-raid6 topologies are extremely unreliable. The btrfs wiki itself describes btrfs parity RAID as "[mostly implemented](https://btrfs.wiki.kernel.org/index.php/FAQ#Can_I_use_RAID.5B56.5D_on_my_Btrfs_filesystem.3F)," and it explicitly recommends "[btrfs] parity RAID [should] be used only for testing purposes."

https://www.phoronix.com/news/Btrfs-Warning-RAID5-RAID6
It remains impossible to use BTRFS as RAID 5/6 solution.

The issues are so severe that the biggest NAS company Synology decided to keep BTRFS as a filesystem but decided to use Linux RAID (!) on its systems.
https://kb.synology.com/en-uk/DSM/tutorial/What_was_the_RAID_implementation_for_Btrfs_File_System_on_SynologyNAS

[ryecoaaron]

OMV doesn't even allow you to create raid5/6 with btrfs but things are getting better - https://www.phoronix.com/news/Linux-6.2-Btrfs-EXT4

but what I used to love about OMV was always the great flexibility to freely choose filesystems and RAID management solutions. Now it all seems to go into BTRFS support as a filesystem, and especially as a RAID solution.

Nothing has changed with ext4 and xfs. You can still use mdadm. So, you can still freely choose. The snapshot stuff can't be implemented for the filesystems because they don't support it.

[MarcS1975]

things are getting better - https://www.phoronix.com/news/Linux-6.2-Btrfs-EXT4
That's just Marketing. Thigs are still not getting better say users:
https://www.reddit.com/r/btrfs/comments/127zmsx/what_do_you_think_about_the_kernel_62_btrfs/

OMV doesn't even allow you to create raid5/6 with btrfs
Well some of your Moderators in the OMV Forum don't know this. They advise that "BTRFS replaces SnapRaid. "

[ryecoaaron]

The first user in your reddit thread says

Now, if you can endure all that, you technically can use RAID5 now.

How is that not better?

Well some of your Moderators in the OMV Forum don't know this. They advise that "BTRFS replaces SnapRaid. "

"My" mods? That technically isn't wrong. btrfs can calculate parity and protect against bitrot.

What is the goal of all of this for you? Do you want all btrfs removed?

[votdev]

I commend your enthusiasm to focus on one filesystem and add more Plugin functionality for OMV, but what I used to love about OMV was always the great flexibility to freely choose filesystems and RAID management solutions. Now it all seems to go into BTRFS support as a filesystem, and especially as a RAID solution.

This is wrong. You can still use whatever file system you want. The only thing is that OMV fully integrates Btrfs feature support into the UI. Nothing more. You can still choose. That snapraid does not handle subvolumes correct is a bad situation, but there are workflows which will enable you to still use it.

[MarcS1975]

What is the goal of all of this for you? Do you want all btrfs removed?

Of course not! Nothing shall be removed. Its a filesystem option for anyone to try out. I use it myself as a filesystem (not as RAID). So it will always require some kind of RAID on top of BTRFS. And my surprise was just, that the the new OMV functionality makes it impossible to use some popular RAID solutions. So my "Goal" was to find a compromise (the tick box), so users can continue to use their legacy OMV systems based on Snapraid. I understand that this is not going to happen, and we have to live with 'workarounds'. Fine.
So then my '2nd Goal' is to learn and understand what brought you guys to favour BTRFS so much, when it is so widely dismissed as an unfinished and unreliable solution. If you say, you disagree and not interested in sharing reasons then that also fine.
:)

[votdev]

So then my '2nd Goal' is to learn and understand what brought you guys to favour BTRFS so much, when it is so widely dismissed as an unfinished and unreliable solution. If you say, you disagree and not interested in sharing reasons then that also fine. :)

ZFS will never go into the Linux kernel, so Btrfs is the way to go. If you prevent RAID5 and RAID6, then you should be fine. Btrfs is the default file system in many Linux distributions nowadays, so it might not be too bad. Second, the main feature i wanted to have in OMV was the ability to create snapshots of shared folders to allow easy recovering of previous states. This is super easy with Btrfs. I don't think there must be more reasons to switch to Btrfs.

[ryecoaaron]

Personally, I only favor btrfs over zfs. And that is only because btrfs is easier to support codewise and people wise. my first choice in filesystems is still ext4 but I use ext4,xfs, and btrfs with my backups just in case a bad bug hits one of the filesystems. I dont run any kind of raid anymore and prefer borgbackup to snapraid or other bitrot detection systems.

[MarcS1975]

i wanted to have in OMV was the ability to create snapshots of shared folders to allow easy recovering of previous states.
This is super easy with Btrfs.
Agree! And that's the only reason I use BTRFS, even at the cost of having to put up with a RAID on top of it.

[MarcS1975]

I dont run any kind of raid anymore and prefer borgbackup to snapraid or other bitrot detection systems.
I see. Have you ever compared the space requirements of borgbackup vs snapraid? Borg is very efficient but my assumption is that Borg still use a lot more space than Snapraid's single parity.

[ryecoaaron]

Have you ever compared the space requirements of borgbackup vs snapraid? Borg is very efficient but my assumption is that Borg still use a lot more space than Snapraid's single parity.

No. But on a very important backup, all of the source files (sql dumps, images, source code, etc) consume about 383GB of space. The borgbackup directory with 26 backups (8 hourly, 7 daily, 4 weekly, 5 monthly, 2 yearly) consumes 142 GB using dedupe and compression. This same setup is running on three servers with two disks each. It gives me the ability to restore from 26 different times easily just by picking an archive.

[MarcS1975]

So if a disk fails, instead of replacing a disk and letting the RAID do the resilvering work, you replace the disk and then manually select the archives to restore to the new disk?

[ryecoaaron]

No. All three servers are backup servers. Being highly available is not important for the backup servers. If a disk on a backup server with the primary backup (rsync'd from the primary server) fails, I just replace the disk and the scheduled rsync will fix it. If the disk with the borgbackup fails, I would rsync the borgbackup from one of the other backup servers to it. The primary server is running two nvme sticks in raid 1. I also have offsite backups.