Pool usage misreported

[sfranzen]

While testing the coloured progress bars for the dashboard, I filled one pool ("pool1") entirely with data:

[root@rockstorvm share1]# dd if=/dev/zero of=data bs=1M count=2k
dd: error writing ‘data’: No space left on device
1808+0 records in
1807+0 records out
1894776832 bytes (1.9 GB) copied, 5.78675 s, 327 MB/s

The result is that, although the pool cannot store more data, it doesn't look alarmingly full (a red colour is triggered above 90%):

It happens because the pool reports the size of its file contents, while some space is also reserved by btrfs:

[root@rockstorvm share1]# btrfs fi show .
Label: 'pool1'  uuid: 6609efe9-6275-427c-baa9-bb0fa8528d63
        Total devices 1 FS bytes used 1.77GiB
        devid    1 size 2.00GiB used 2.00GiB path /dev/sdb

This example is a bit contrived with my tiny virtual disks, but nonetheless I would like the dashboard to show accurately when a pool is nearly full and therefore in need of management, so I'll go have a look at the pool code.

Perhaps in the future it would also be nice to display the btrfs data/metadata in more detail, on the individual pool views.

[sfranzen]

Ok, after some investigation it turns out we have two issues:

• The space allocated for btrfs metadata and system data (i.e. not necessarily fully used) is not reported as used space, even though it is not available for regular data;
• The total size reported for some RAID configurations is not correct, especially with unequal disk sizes.

Luckily an algorithm is available that should work in all of these cases. The author posted it to the btrfs mailing list a while ago and also has a javascript implementation on his btrfs space allocator website. I've translated it to Python and will soon test it on some virtual disk arrays.