Permalink
Browse files
Make heatmap better scriptable for flexibility
Instead of ending up with feature creep in the command line parsing,
make the program better scriptable, so users can build whatever they
want.
This commit completes moving most of the code from the main() function
away to reusable building blocks.
When doing something more sophisticated than just creating a picture of
a complete filesystem or a single block group, it's better to do it from
python and use the btrfs library to find the objects that we want to
display.
The heatmap code is not a full-blown library, but that doesn't prevent
us from importing the heatmap.py from another script in the same
directory and then using functions from it:
First you need the imports:
import btrfs
import heatmap
1. Equivalent of doing heatmap.py -o full-fs.png /
fs = btrfs.FileSystem('/')
heatmap.walk_dev_extents(fs).write_png('full-fs.png')
output:
scope device 1
grid order 7 size 10 height 128 width 128 total_bytes 255057723392 bytes_per_pixel 15567488.0
pngfile full-fs.png
2. Generate an image of the four newest block groups together
four_newest_bg = [fs.block_group(chunk.vaddr)
for chunk in list(fs.chunks())[-4:]]
grid = heatmap.walk_extents(fs, four_newest_bg)
parts = ['fsid', fs.fsid, 'startat', four_newest_bg[0].vaddr]
png_filename = heatmap.generate_png_file_name(parts=parts)
grid.write_png(png_filename)
Note that I use list() on the output of the chunks, to be able to slice
it. The chunks() function returns a generator, and by converting it to a
list the complete content is retrieved immediately. After that I take
the four block groups with the highest vaddr, which are the most
recently generated ones.
output:
scope block_group 320029589504 321103331328 322177073152 323250814976
grid order 10 size 10 height 1024 width 1024 total_bytes 4294967296 bytes_per_pixel 4096.0
pngfile fsid_ed10a358-c846-4e76-a071-3821d423a99d_startat_320029589504_at_1482095269.png
This example uses the png file name generator helper which adds a
timestamp so we can easily repeat it to get a timelapse. I'm going to
use this very soon for free space fragmentation and autodefrag behaviour
debugging on a problematic filesystem.
3. Show usage, separate image per device, more verbose output
fs = btrfs.FileSystem('/mnt/raid0')
for device in fs.devices():
heatmap.walk_dev_extents(fs, [device], verbose=1).write_png('device_%s.png' % device.devid)
output:
scope device 1
grid order 5 size 10 height 32 width 32 total_bytes 26843545600 bytes_per_pixel 26214400.0
dev_extent devid 1 paddr 20971520 length 8388608 pend 29360127 type SYSTEM|RAID0 used_pct 0.10
dev_extent devid 1 paddr 29360128 length 1073741824 pend 1103101951 type METADATA|RAID0 used_pct 6.57
dev_extent devid 1 paddr 1103101952 length 1073741824 pend 2176843775 type DATA|RAID0 used_pct 50.51
pngfile device_1.png
scope device 2
grid order 5 size 10 height 32 width 32 total_bytes 26843545600 bytes_per_pixel 26214400.0
dev_extent devid 2 paddr 1048576 length 8388608 pend 9437183 type SYSTEM|RAID0 used_pct 0.10
dev_extent devid 2 paddr 9437184 length 1073741824 pend 1083179007 type METADATA|RAID0 used_pct 6.57
dev_extent devid 2 paddr 1083179008 length 1073741824 pend 2156920831 type DATA|RAID0 used_pct 50.51
pngfile device_2.png
The signatures of the used functions are:
1. Working with devices:
walk_dev_extents(fs, devices=None, order=None, size=None,
default_granularity=33554432, verbose=0)
* fs is the btrfs.FileSystem object
* devices is a list of device objects, or None to use all of them
* for order and size, see the usual --help and documentation
* default_granularity defines the amount of bytes that should be mapped
(approximately) on a single pixel in the output. By default, this is
32MiB
* a higher number for verbose makes the output more verbose (like -vvv
on the command line of heatmap.py would be verbose=3)
2. Working with block groups:
walk_extents(fs, block_groups, order=None, size=None,
default_granularity=None, verbose=0)
* block_groups is a list of one or multiple block group objects
* for block group internals, default_granularity defaults to the sector
size of the filesystem, which is often 4096 bytes
3. A helper for generating file names:
generate_png_file_name(output=None, parts=None)
* parts is a list of filename parts that will be concatenated, after
which a timestamp is also added.
e.g. parts=['foo', 'bar'] => foo_bar_at_1482095269.png
* output can be a filename, in which case the function just returns
that filename again
* output can be a directory, in which case the function will return a
path to an autogenerated filename using parts in that directory- Loading branch information
