Lookup orphaned subvolume ids

When deleting a subvolume, it gets cleaned up in the background. Already
deleted subvolumes that are not completely gone yet are registered in
the root tree under the ORPHAN_OBJECTID, which is -5, which is actually
18446744073709551611, almost at the end of the tree space.

show_orphaned_subvols.py is a little example to show how you can show
them on your filesystem.

To see some output, you can do something like this:

watch ./show_orphaned_subvols.py /

and then in a second terminal:

mkdir orphan
cd orphan
for i in $(seq 1 100); do btrfs sub snap / $i; done
btrfs sub del *

It will show a list of subvolume ids for a short time, like:

[309, 412, 413, 417, 420, ...]

`btrfs sub list` has similar functionality with the `-d` option, but it
iterates over all existing subvolumes and then filters them for a
deleted flag, which is a horrible way to look them up if you have a
tens of thousands of subvolumes (snapshots) and are trying to remove a
few hundred up to a few thousand of them in one go.

`btrfs sub sync` has a more clever implementation that retrieves the
list with orphan subvolume ids once, and then does a search for those
ids in the regular subvolume list in the root tree to see if any of them
is still there. It exits if all of them are gone.

At $work, we run some filesystems with insane amounts of subvolumes
(like ~80k) for backup purposes, and we need to throw away between one
and two thousand expired snapshots every day. When doing so, it's
advised to not just submit the deletion of all of those at once, but
throttle it a bit. I cannot use `btrfs sub sync` for that, because I
want to delete new subvolumes everytime the amount of them that are
being cleaned gets below a certain treshold.

The lookup for orphans is very cheap, so it can be used to create a
procedure like this:

    remove_snapshots = ['list', 'of', 'subvolume', 'paths', ...]
    concurrent = 25
    for cur in xrange(len(remove_snapshots)):
        while True:
            amount_orphans = len(fs.orphan_subvol_ids())
            if amount_orphans < concurrent:
                break
            time.sleep(1)
        print("[{0}/{1}] removing {2}".format(
            cur+1, amount_orphans, remove_snapshots[cur]))
        your_favourite_way_to_btrfs_sub_del(remove_snapshots[cur])

Moo! Tune the numbers by looking at how your filesystem and hardware
behaves by upping the concurrent number so that you're just hitting
enough IO to get it over with quickly and not make it unresponsible.

btrfs/ctree.py

@@ -24,11 +24,20 @@
 import uuid
 
 
+def ULL(n):
+    if n < 0:
+        return n + (1 << 64)
+    return n
+
+
 DEV_ITEMS_OBJECTID = 1
+ROOT_TREE_OBJECTID = 1
 EXTENT_TREE_OBJECTID = 2
 CHUNK_TREE_OBJECTID = 3
 FIRST_CHUNK_TREE_OBJECTID = 256
+ORPHAN_OBJECTID = ULL(-5)
 
+ORPHAN_ITEM_KEY = 48
 BLOCK_GROUP_ITEM_KEY = 192
 DEV_ITEM_KEY = 216
 CHUNK_ITEM_KEY = 228
@@ -176,6 +185,14 @@ def block_group(self, vaddr):
                         btrfs.ioctl.search(self.fd, tree, min_key, max_key, nr_items=1)]
         return block_groups[0]
 
+    def orphan_subvol_ids(self):
+        tree = ROOT_TREE_OBJECTID
+        min_key = Key(ORPHAN_OBJECTID, ORPHAN_ITEM_KEY, 0)
+        max_key = Key(ORPHAN_OBJECTID, ORPHAN_ITEM_KEY, ULLONG_MAX)
+        subvol_ids = [header.offset
+                      for header, data in btrfs.ioctl.search(self.fd, tree, min_key, max_key)]
+        return subvol_ids
+
 
 class Device(object):
     dev_item = struct.Struct("<3Q3L3QL2B16s16s")

examples/show_orphaned_subvols.py

@@ -0,0 +1,8 @@
+#!/usr/bin/python
+
+from __future__ import print_function
+import btrfs
+import sys
+
+fs = btrfs.FileSystem(sys.argv[1])
+print(fs.orphan_subvol_ids())