BtrFS

BtrFS

About

BtrFS is a next-generation, "copy on write" (aka "CoW") file system that supports numerous features such as:

• Copy-on-write crash resiliency, and file system consistency
• Advanced volume and subvolume management
• Reflinks and snapshots ("zero byte" copies of existing files with none of the disadvantages of symlinks/hardlinks)
• Real time lightweight transparent compression (compress blocks on disk, invisible to applications)
• Deduplication - identify identical chunks of data even within files, and safely remove duplicate data transparently (without the enormous memory overhead of ZFS - BtrFS dedup can even run on a Raspberry Pi).
• Real time, lightweight checksum checking of all data and metadata
• On-demand "scrub" operations to verify the validity of all used data on a volume
• Advanced RAID and data duplication options, including the ability to use RAID efficiently on non-matching drives
• Instant RAID creation - no long waits to generate large RAID sets
• Ability to hot-convert between RAID levels, add in new disks, resize, re-balance data, etc

The official BtrFS project page including status and command references can be found here:

• https://btrfs.wiki.kernel.org

There are currently 24 unique entities from approximately 10 large companies all contributing to BtrFS. It is not under control or at the mercy of any one single company, entity or individual. BtrFS is included in the Linux kernel and distributed with all modern Linux systems.

Why BtrFS for RetroNAS?

Many file systems exist in Linux, and the platform can be somewhat overwhelming when it comes to choice of file systems.

Older, traditional file systems like ext4 are often the default file system for Linux systems (ext4 is the default root filesystem for Raspberry Pi OS). On Windows, NTFS is similar. While these are "tried and true", they suffer the glaring issue of not being crash safe - i.e.: on hard power down without cleanly shutting down your operating system, the file system needs to be checked ("checkdisk" or "fsck" style operations) on startup. This is fine for a small <1TB system drive, but on multi-TB NAS drives, this can be painful.

Some file systems are attempting to extend their legacy functionality to modern feature sets. XFS is such a file system that is attempting to migrate from legacy data modes to more modern CoW (copy-on-write) systems, however is still far behind BtrFS in terms of complete features.

ZFS has long been considered the gold standard of reliable file systems, however has a considerable CPU and RAM overhead, as it is designed for larger systems. While excellent for multi-drive systems running on powerful Intel CPUs with it's extremely safe RAIDZ family of data management, ZFS runs poorly on a smaller platform like a Raspberry Pi. ZFS also does not lend itself well to dynamically resizing and reshaping arrays, where you may want to add in ad-hoc, mis-matching drives at a later date.

ZFS's deduplication algorithm currently requires the checksum database used to identify identical blocks to exist in RAM, which is VERY RAM heavy.

BtrFS solves some of these issues - it can dynamically resize and reshape it's RAID configuration, including changing RAID levels, as well as efficiently use multiple mis-matching hard disks with far less space wastage (sometimes none at all) compared to ZFS or traditional RAID. BtrFS has the ability to background scan a file system and store identical checksums in an SQLite database on-disk, almost entirely removing the RAM requirement for deduplication.

In a multi-drive setup, BtrFS's "RAID1" algorithm duplicates all chunks of data and metadata. Across 3 mismatching (different sized) drives, BtrFS efficiently offers N-1 safety of data without wasting any drive space.

With inline transparent compression (zstd recommended for Intel systems, lzo recommended for ARM and RPi systems), and deduplication, BtrFS can save around 20% of disk space transparently - ideal for systems or emulators that cannot access file-compressed data such as zip or chd files.

How to use BtrFS - single disk

When formatting a single disk for your RetroNAS top level share using Cockpit, simply choose "BtrFS" as your file system instead of other Linux file systems such as ext4 or XFS.

Mount options and compression

You can add in mount options at any time, no need to re-format the drive. For example, adding in the mount options:

compress=lzo

Will tell BtrFS to compress all data transparently (i.e.: the data on disk is compressed, but will appear uncompressed to you and any applications using the disk). BtrFS uses "opportunistic compression", where it attempts to compress the first few KB of data. If this cannot compress well (say, the file is already compressed, like an MP4 or ZIP file), BtrFS will not bother compressing the rest. You can override this behaviour if you like with the option:

compress-force=lzo

Likewise other compression algorithms are available. You can use zstd instead of lzo to use the excellent ZStandard compression library. However this uses more CPU and is probably not suited to a Raspberry Pi. You can switch compression algorithms and styles any time you like by changing the mount option and rebooting. BtrFS stores the compression type for each chunk of data in metadata, so it will happily read files compressed in one algorithm even if you change to another algorithm later. New and updated files will be written in the new compression algorithm.

To use multiple mount options, separate them with a comma and no spaces. For example:

compress=lzo,noatime

Will tell BtrFS to use compression on the file system, as well as never store "atime" (access time) data on disk. This is very useful to reduce file IO on low end systems like Raspberry Pis where you may not care to store the timestamp on disk of every file read or access.

Checking compression levels

To see how well data is being compressed, use the compsize tool from the command line. This tool requires "sudo" access to calculate the values. Navigate to a specific directory, and run the command on a given file:

sudo compsize Shadow_of_the_Colossus_SCUS_974.72.iso 

Type       Perc     Disk Usage   Uncompressed Referenced  
TOTAL       54%      1.4G         2.7G         2.5G       
none       100%      261M         261M         6.0M       
zstd        49%      1.2G         2.4G         2.5G       

Here we can see a PS2 ISO. It's 2.7GB in size, although only 2.5GB "referenced" due to deduplication (this will be covered later). In the "TOTAL" row, we can see that 2.5GB has been compressed down to 1.4GB for a 54% compression ration (smaller % numbers are better).

The line below that says "none" tells us how much of the data was uncompressible. Here 261M was uncompressible (although that was shrunk to 6MB thanks to deduplication, which we will cover later).

And finally, the "zstd" row tells us that a total of 2.5GB was compressed down to 1.2GB using zstd specifically, resulting in that compressible data being 49% of its original size (smaller % numbers are better).

So any high level tool such as ls or checking this file via Samba in Windows using right-click -> properties will report the file as the full 2.7GB. However, on-disk, only 1.4GB of space is actually consumed. To all external systems such as Samba, OpenPS2Loader, etc, the file looks normal, and no "unzip" style operation is required to access the contents. BtrFS handles all of this transparently in the background.

This can be repeated for an entire directory. For example:

cd /data/retronas/ps2/OpenPS2Loader/DVD
sudo compsize .

Processed 50 files, 700815 regular extents (897123 refs), 1 inline.
Type       Perc     Disk Usage   Uncompressed Referenced  
TOTAL       79%       89G         111G         128G       
none       100%       35G          35G          34G       
zstd        70%       53G          76G          94G       

Depending on physical disk speed this operation can take several minutes if it's performed on a very large data set.

How to use BtrFS - multi disk and RAID

TBA. Note that extra disks can be added after a single drive has been formatted and mounted as BtrFS. So feel free to use one disk, and this guide will be updated soon to demonstrate how to extend to multiple disks. Likewise BtrFS can convert data to and from any native RAID format, so a single drive can be used at first and then extended to RAID or multi-disk at a later time.

Deduplication

TBA

BtrFS in Windows

You can mount a BtrFS formatted drive in Windows. This is useful if you want to connect the drive directly and manage the contents, or utilise faster USB3 connectivity (4.8Gbit/s max, depending on drive speed) on a desktop computer to transfer initially large data sets before migrating the drive to a 1GbE Ethernet NAS.

WinBtrFS supports WindowsXP and up.

Install the WinBtrFS driver on Windows, set your mount options in the Windows system registry, reboot, and then attach the drive:

• https://github.com/maharmstone/btrfs

Ensure you read the section on mount options. You will need to set compression algorithms to match what you intend to use on your RetroNAS device. These will need to be set in the Windows system registry before you mount your drive:

• https://github.com/maharmstone/btrfs#Mount-options