Permalink
Browse files
Implement logical to inode v2 ioctl
The v2 LOGICAL_INO ioctl allows us to set a flag 'ignore_offset', which results in returning the inodes that reference any part of the extent. It also allows us to provide a larger buffer to receive results, so that it can handle extents which have more than 2730 references. This new ioctl will be included in linux 4.15. Relevant commits: * commit c995ab3 "btrfs: add a flag to iterate_inodes_from_logical to find all extent refs for uncompressed extents" * commit d24a67b "btrfs: add a flags argument to LOGICAL_INO and call it LOGICAL_INO_V2" * commit b115e3b "btrfs: increase output size for LOGICAL_INO_V2 ioctl" Also see the commit message on 481a4f9 here in python-btrfs, which shows the previous limitation. ------------------------------------- 8< ------------------------------------- Here's the testing scenarios I used earlier (from btrfs mailing list): -# cp /boot/vmlinuz-4.14.0-rc1-zygo1 /btrfs -# ./show_block_groups.py /btrfs block group vaddr 0 length 4194304 flags SYSTEM used 16384 used_pct 0 block group vaddr 4194304 length 8388608 flags METADATA used 131072 used_pct 2 block group vaddr 12582912 length 8388608 flags DATA used 0 used_pct 0 block group vaddr 20971520 length 268435456 flags METADATA used 0 used_pct 0 Using 'v1': -# ./show_block_group_data_extent_filenames.py 12582912 /btrfs block group vaddr 12582912 length 8388608 flags DATA used 4198400 used_pct 50 extent vaddr 12582912 length 4198400 refs 1 gen 17 flags DATA root 5 inode 258 offset 0 path utf-8 vmlinuz-4.14.0-rc1-zygo1 Let's overwrite the first few blocks: -# dd if=/dev/urandom bs=16K conv=notrunc of=/btrfs/vmlinuz-4.14.0-rc1-zygo1 count=1 1+0 records in 1+0 records out 16384 bytes (16 kB, 16 KiB) copied, 0.000191925 s, 85.4 MB/s Here we see the limitation of the 'v1' ioctl. I get no name back for the first extent any more: -# ./show_block_group_data_extent_filenames.py 12582912 /btrfs block group vaddr 12582912 length 8388608 flags DATA used 4214784 used_pct 50 extent vaddr 12582912 length 4198400 refs 1 gen 17 flags DATA extent vaddr 16781312 length 16384 refs 1 gen 19 flags DATA root 5 inode 258 offset 0 path utf-8 vmlinuz-4.14.0-rc1-zygo1 (And to test that the 'v1' also still works...) When I change it to use LOGICAL_INO_V2, it shows the same: -# ./show_block_group_data_extent_filenames2.py 12582912 /btrfs block group vaddr 12582912 length 8388608 flags DATA used 4214784 used_pct 50 extent vaddr 12582912 length 4198400 refs 1 gen 17 flags DATA extent vaddr 16781312 length 16384 refs 1 gen 19 flags DATA root 5 inode 258 offset 0 path utf-8 vmlinuz-4.14.0-rc1-zygo1 When I also set the ignore_offset flag: -# ./show_block_group_data_extent_filenames2.py 12582912 /btrfs block group vaddr 12582912 length 8388608 flags DATA used 4214784 used_pct 50 extent vaddr 12582912 length 4198400 refs 1 gen 17 flags DATA root 5 inode 258 offset 16384 path utf-8 vmlinuz-4.14.0-rc1-zygo1 extent vaddr 16781312 length 16384 refs 1 gen 19 flags DATA root 5 inode 258 offset 0 path utf-8 vmlinuz-4.14.0-rc1-zygo1 Now, for the amount of results we can retrieve, let's cause an extent to have more than 2730 references: /btrfs 2-# btrfs sub create 0 Create subvolume './0' /btrfs 2-# cp /boot/vmlinuz-4.14.0-rc1-zygo1 0/0 /btrfs 2-# for i in $(seq 1 499); do cp --reflink 0/0 0/$i; done /btrfs 2-# for i in $(seq 1 5); do btrfs sub snap 0 $i; done Create a snapshot of '0' in './1' Create a snapshot of '0' in './2' Create a snapshot of '0' in './3' Create a snapshot of '0' in './4' Create a snapshot of '0' in './5' -# ./show_block_groups.py /btrfs block group vaddr 0 length 4194304 flags SYSTEM used 16384 used_pct 0 block group vaddr 4194304 length 8388608 flags METADATA used 507904 used_pct 6 block group vaddr 12582912 length 8388608 flags DATA used 4198400 used_pct 50 block group vaddr 20971520 length 268435456 flags METADATA used 0 used_pct 0 -# ./show_block_group_contents.py 12582912 /btrfs block group vaddr 12582912 length 8388608 flags DATA used 4198400 used_pct 50 extent vaddr 12582912 length 4198400 refs 500 gen 25 flags DATA inline extent data backref root 257 objectid 262 offset 0 count 1 inline extent data backref root 257 objectid 277 offset 0 count 1 inline extent data backref root 257 objectid 288 offset 0 count 1 [...] extent data backref root 257 objectid 663 offset 0 count 1 extent data backref root 257 objectid 366 offset 0 count 1 extent data backref root 257 objectid 715 offset 0 count 1 extent data backref root 257 objectid 306 offset 0 count 1 extent data backref root 257 objectid 470 offset 0 count 1 [...] Total 500 lines, the extra 2500 files in the snapshots are hidden behind the shared metadata refs now... >>> import btrfs >>> fs = btrfs.FileSystem('/btrfs') Checking that 'v1' still works: >>> inodes, bytes_missed = btrfs.ioctl.logical_to_ino(fs.fd, 12582912, 65536) >>> len(inodes) 2730 >>> bytes_missed 6480 Yes, we only get 2730, as expected with a 64k buffer. v2 can do the same: >>> inodes, bytes_missed = btrfs.ioctl.logical_to_ino_v2(fs.fd, 12582912, 65536) >>> len(inodes) 2730 >>> bytes_missed 6480 The bytes_missed is really useful, because it tells us the exact size of the buf we need instead :) >>> inodes, bytes_missed = btrfs.ioctl.logical_to_ino_v2(fs.fd, 12582912, 65536 + 6480) >>> len(inodes) 3000 >>> bytes_missed 0 Yay!
- Loading branch information