Skip to content

fdu 0.6.0

Latest

Choose a tag to compare

@jlevy jlevy released this 12 Nov 22:29
Immutable release. Only release title and notes can be modified.
4450be7

fdu 0.6.0 finds what was written recently, from one scan with no saved state, and shows how full the disk is beside what the tree holds. Every time in a report is read on one clock, change time by default, which catches copies and extracted archives that modification time misses.

This release renames the time flags and machine-output fields, changes the report schema to fdu.report/12 and the watch stream to fdu.stream/3, and changes parts of the Rust and Python APIs. fdu is alpha software, so the old spellings are gone rather than kept as aliases. See Upgrading from 0.5.0 before upgrading a script, a parser, or a library caller.

What’s Changed

Features

  • What was written recently. fdu PATH --kind file --since 24h shows where the data written in the last day now lives, by folder and largest first, and fdu PATH --view largest --since 24h lists the largest of those files. --since is inclusive and --before exclusive; each takes an age such as 2h or 30d, or an RFC 3339 instant. Keep --kind file: without it, a directory with any activity in the window counts at its full size. The answer is data written, not net growth: a file written in the window counts in full, and deleted files do not appear.

  • One clock for every time, change time by default. A report reads one clock for its window, --sort time, the tree’s age column, --long, recent, and ages, so it never mixes two. The default is change time: the newer of an entry’s modification time and its status change, which the operating system sets whenever data or metadata is written. It catches data that arrived carrying an older modification time, as cp -p, rsync -a, Finder and Explorer copies, mv from another filesystem, extracted archives, and Cargo’s crate sources do. Its errors point the safe way for both questions a time answers. Asked what is new, it can over-include, since a chmod looks new, rather than omit a copy. Asked what is old, it does not call a just-restored backup stale.

  • --time modified when change time says too much. It reads modification time for the whole report, as ls -l shows. Use it after a bulk chmod, chown, or relabel, and in a tree whose files are hard-linked from elsewhere. Making or removing any hard link moves the change time of every other name of the file, which uv and pnpm installs from a shared store, cp -l, and git clone --local do routinely: after uv 0.12.1 installed the same two packages into a second environment on ext4, 26 files (632 KiB) of the first read as newly written, and none did under --time modified. On Windows, a file fdu could list but not open has no observed status change; it is timed by its modification time, and a note counts such files.

  • --view ages: when the space arrived. The selected regular files are counted in nine fixed bands on the report’s clock, newest first, from future and under 1h to over 3y:

    $ fdu / --one-filesystem --view=ages
       7.7 GiB   18.9%  under 1h     9,004 files (7.7 GiB gitignored)
        25 GiB   61.0%  1h to 1d     93,255 files (18 GiB gitignored)
       3.8 GiB    9.4%  7d to 30d    61,112 files (768 MiB gitignored)
       984 MiB    2.3%  30d to 90d   12,810 files
       551 MiB    1.3%  90d to 1y    7,147 files
       2.0 GiB    5.1%  1y to 3y     54,225 files (66 MiB gitignored)
       848 MiB    2.0%  over 3y      37,542 files (24 KiB gitignored)

    The bands add up to --view summary for any selection. Where the selection admits files alone, each band is also what --kind file with the window it names selects. The view reads metadata only, shares one walk with other views (--view tree,ages --kind file --since 7d), and is part of --view full. Machine output keeps every band, with its edges as ages and as instants and a cumulative tally of it and every newer band.

  • --view volumes: how full the disk is. Each filesystem the report touches gets three lines: its capacity, share in use, path, and type; what is available; and what this report counted there.

    $ fdu ~/.cargo --view=volumes
       251 GiB   96% in use  .  (ext2/3/4)
                             10 GiB available
                             318 MiB counted by this report

    A walk without --one-filesystem shows each mount below the path as its own row, and fdu PATH --view volumes --scan-depth 0 reads capacity alone. Capacity is observed when the report is read and never cached, so --stale-ok shows current capacity beside a stale tree. Reading it mounts nothing the tree view would not: below the root, a filesystem is asked only where the walk listed it and, on Linux and macOS, the mount table shows a mounted filesystem, not an automount trigger, at exactly that path. The table is read once per report, so an automount that expires between that read and the call there can be mounted again, in a window as long as the calls before it. Anything else keeps its row and its counts with capacity unknown and the reason. Rows never add up: an APFS, btrfs, or ZFS row says its available space is shared within its pool, and snapshots and deleted files still held open use space no walk sees. The view was exercised on Linux, real automounts included. On macOS and Windows it is covered by automated tests, and its output on real APFS and NTFS volumes has not been checked by hand.

  • Selection coverage. A report whose selection narrows the scan ends with a note such as note: selected 2,951 of 27,060 files, 704 MiB of 1.8 GiB, so a filtered root’s 100% is not read as the whole tree. Machine output carries both as selected and scanned on every report.

  • Both clocks in machine output. request.time names the report’s clock, and every list row gives time_ns on that clock beside the raw mtime_ns and changed_ns. An arrival that kept old content times is a row whose changed_ns is recent and whose mtime_ns is old. Watch upserts carry changed_ns too.

  • Python. Selection(since=, before=, time=TimeBasis.MODIFIED), SortKey.TIME, View.AGES, and View.VOLUMES mirror the command line, with typed AgesSection and VolumesSection results and Report.selected and Report.scanned. Index.freshness reads a retained index’s freshness without building a report.

Fixes

  • Automount points on Linux. Under --one-filesystem, a kernel automount point that is not mounted yet, such as an NFS nested export or tracefs under debugfs, is now a boundary on glibc builds, which the wheels are. 0.5.0 listed it on a first run, which mounted the other filesystem and counted the files at its top, and stopped at it on the next. A directory the walk enters on another device than its parent, such as a direct automount trigger or a kernel automount point the walk itself mounts, is also recorded as the filesystem found there, so a first run and the next agree about it. A browsable key of an indirect autofs map is not yet: it is on its parent’s device until listed, so it is still recorded as the stat found it, and the two runs can disagree about it.
  • Python roll-ups. Index.total() and Index.rollup() take a roll-up and its provenance from one native read. They made two, so a refresh between them could pair a roll-up with the provenance of a later state.
  • Python opened reads. OpenedIndex.read raises TypeError for an argument that is not a projection, where 0.5.0 read it as Diagnostics.
  • Incomplete-subtree note. note: incomplete subtrees remain visible below the size threshold is printed only when the tree shows such a row. 0.5.0 printed it whenever a share threshold applied and some directory was not listed in full, even when the selection gave that directory no row, as --kind file does for an unreadable directory, or when --depth, --breadth, or --limit hid the row.

Guidelines and Content

  • Agent skill. The installed skill shows how to find where recently written data is using disk space, explains the change clock and when --time modified is the one to use, and covers ages, volumes, the coverage note, fdu.report/12, and fdu.stream/3. fdu --help and fdu --docs describe the same commands. The skill’s recipe for recent writes is one walk, fdu PATH --kind file --since 24h --view tree,summary,largest, which gives the folders, their total, and the largest files. To ask more of the same tree without walking it again, add --cache on to that command and --stale-ok to each later one, which answers from the first walk’s snapshot and labels the answer stale. Give every command the same --cache-dir, outside the scanned path, since a snapshot inside it is itself a file written in the window. Re-run fdu --install-skill after upgrading, since a skill names the version that generated it.

Documentation

Upgrading from 0.5.0

  • Flags: --modified-since and --modified-before are --since and --before, and --sort mtime is --sort time. The old spellings are refused: each exits 2 with one line naming its replacement and the clock that changed with it, such as fdu: --modified-since was renamed --since in 0.6.0, and the window reads change time unless --time=modified is given. --sort time with only types, extensions, families, languages, or documents is refused as well, since their rows carry no time; 0.5.0 accepted --sort mtime there and ordered them by name. Beside a view that has a time, they keep size order and a note says so. With only summary, ages, or volumes, whose order is fixed, it is refused too.
  • Ages and windows read change time: the tree’s age column, --long, recent, and every window give different answers for copied, restored, re-permissioned, and hard-linked files. A tree copied in yesterday reads 1d however old its files, so --before 30d no longer selects it. Add --time modified for the answers 0.5.0 gave.
  • Text output: a filtered report adds a note: selected … line on standard error, which --quiet hides.
  • --view full: it now includes ages, right after summary, so read machine sections by their view, not their position. It leaves volumes out, because capacity describes the host and not the tree: ask for --view volumes or --view tree,volumes.
  • Machine output: accept fdu.report/12 explicitly. On tree nodes and totals, mtime_ns, newest_mtime_ns, and modified_at are time_ns, newest_file_time_ns, and time_at; a summary’s newest_mtime_ns is newest_file_time_ns; and a list row’s modified_at is time_at. A list row keeps mtime_ns, always its modification time, and gains time_ns and changed_ns; age_ns is the age of time_ns. Every report gains request.time, selected, and scanned. The cache schema remains fdu.cache/3. See the machine-output reference.
  • Watch stream: change records carry fdu.stream/3. An upsert gains changed_ns; nothing else in the record changes.
  • Cache: snapshots are format 7, which keeps the device each walk read its root on. fdu 0.6.0 reads nothing 0.5.0 cached: the next run walks and analyzes afresh, --stale-ok fails until a run writes a new snapshot, fdu --cache-status=all lists the old files as stale, and fdu --cache-clear=all removes them.
  • Rust: query::report and query::report_roots take a capacity observation as their last argument: None, or what query::observe_volumes returns for a request that names volumes. Selection::modified is Selection::window, a TimeWindow, beside the new Selection::time; SortKey::Mtime is SortKey::Time; and ReadSpec::modified_since and modified_before are since and before. Rows carry both clocks as Times: FileRow::times, TreeNode::time and newest_file, and SummaryRow::newest_file. ViewSpec and Section gain Ages and Volumes; Report, FileRow, Change, RollUp, and RollUpSummary gain fields; and CONTENT_REPORT_SCHEMA is gone. Update struct literals and exhaustive matches. scan::scan now emits the root itself first, as an upsert of the empty path.
  • Python: Selection(modified_since=, modified_before=) is Selection(since=, before=), and SortKey.MTIME is SortKey.TIME. SortKey("mtime") raises InvalidArgumentError naming time; SortKey.MTIME and the old keyword arguments fail with Python’s own AttributeError and TypeError, which do not say that the clock changed with the name. FileRow.modified_at, TreeNode.mtime_ns, TreeNode.newest_mtime_ns, TreeNode.modified_at, the same TreeTotal fields, and SummaryRow.newest_mtime_ns take the machine-output names above. A hand-built FileRow, Report, ReportRequest, or RollUp takes new required fields. A whole-root Index.refresh() counts the root among unchanged.

The changelog lists the API changes in detail.

Full commit history: v0.5.0 … v0.6.0