Skip to content

fdu 0.2.0

Choose a tag to compare

@jlevy jlevy released this 28 Sep 17:56
· 769 commits to main since this release
Immutable release. Only release title and notes can be modified.
v0.2.0
6ec7716

fdu 0.2.0 makes the command people type cheaper. The default fdu PATH report no longer writes a cache snapshot that no later report reads, a large one-shot index is freed after the answer instead of before it, and a report that asks for several content views resolves each file once. The cache contract is simpler: --cache takes auto, on, or off, and answering from a snapshot without touching the tree is its own flag, --stale-ok. Code analysis gains an aligned code table and population control, and tree output gains display bounds. The same engine still ships three ways: the fdu command line, the fdu Python package, and the fdu-core Rust crate.

This release changes flags, both versioned report and cache-status documents, the Rust and Python cache policy, and the macOS cache location. Read Upgrading from 0.1.0 before updating a script, a machine-output consumer, or a library caller.

Performance

On Linux, fdu was measured against seven other disk-usage tools for the first time, on the same generated 1,000,001-entry tree and paired harness as the macOS comparison, on a quiet 4-vCPU virtualized ext4 host. The two fdu jobs rank differently there:

  • fdu --no-gitignore --view summary is the fastest tool measured, at a 0.94-second median. pdu took 8% longer and diskus 12% longer.
  • The indexed tree is not. Building the reusable exact index and a ten-row tree with the cache off took 1.25 seconds, about 23% longer than pdu and 21% longer than diskus; every other peer was slower. A screen attributes the gap to allocator contention between fdu’s walker threads and its index builder rather than to filesystem work, and that hypothesis is still open.

The Linux comparison has the full matrix, and the cache economics brief says when the cache and the index pay on each platform.

On macOS the comparison was rerun on this release’s engine, on a bare-metal Apple M1 Pro with the same generated tree on internal APFS. fdu built the indexed tree in a 6.4-second median and was faster in paired wall time than every tool measured: dumac, which returns only a total, took 9% longer, pdu 49%, dust 57%, and BSD du about nine times as long. The host was busy, so the quiet gate declared the run uncontrolled; pairing keeps the relative figures, not the absolute seconds. The macOS comparison has the full matrix and the host regime.

Three engine changes in this release were each measured on their own:

  • No unread snapshot. Under --cache auto a one-shot metadata report no longer writes a snapshot. On the million-entry Linux tree the default tree job took 13.8% less wall time on a repeated run (95% interval 10.7% to 16.0% less) and 32.7% less on a first run into an empty cache directory, while a cold-scan placebo stayed within noise (exp-163). On macOS it cut the default tree job’s peak memory 26%, and its 3.1% wall saving was within that host’s noise (exp-165).
  • Index release off the answer path. A one-shot report frees an index of at least 65,536 entries on a detached thread after the answer is complete. Freeing the million-entry index had been 95 ms of a 1.39-second --cache off report, and the default tree job took 3.19% less wall time on Linux (95% interval 1.79% to 4.88% less, exp-160). On macOS, where the kernel dominates each run, the change neither helped nor hurt measurably (exp-164).
  • Multi-view reports resolve each file once. With two or more unfiltered metric views, each regular file’s content record and classification is resolved once for all of them, with output unchanged. On an exploratory, uncontrolled macOS host, 100 four-view report constructions over a 137,085-entry tree ran about 2.5 times faster (exp-159). This is not a scan or end-to-end speedup, and the change is retained provisionally until a quiet rerun settles its inconclusive major-fault gate.

Highlights

  • Cache policy. --cache auto, the default, reads and writes where the kind of run gains from it: content analysis, --watch, and an opened index, not a one-shot metadata report. --cache on also writes after a one-shot report, and --cache off neither reads nor writes. --stale-ok answers from the snapshot alone, says the answer is stale, and fails rather than scanning when there is no snapshot; it replaces --cache only. In Python these are CachePolicy.AUTO, ON, OFF and stale_ok=True; in Rust, CachePolicy::{Auto, On, Off} and Delivery::stale_ok.
  • Code analysis. --analyze code prints one aligned table of code lines, shares, comments, blanks, and analyzed-file coverage, per language and with a bold total that display limits never shrink. Multiline literals, heredocs, Rust lifetimes, and JavaScript regular expressions are counted correctly, which invalidates earlier code-analysis records. --workers sets content-analysis concurrency.
  • Population. --ignored=include|exclude|only chooses which side of .gitignore a report covers. Exclusion prunes safely ignored subtrees and skips their content analysis.
  • Tree display. The default tree goes five levels deep and hides rows under 1% of the selected root. --breadth bounds children per directory, --min-share sets the floor, --full removes every display bound, and a displayed directory always stands for its whole subtree, so the root’s rows and its remainder row add up to the root.
  • Cache location. Snapshots are <key>.metadata.bin with an optional <key>.analysis.bin, under ~/.cache/fdu on macOS and Linux and %LOCALAPPDATA%\fdu on Windows; --cache-dir and FDU_CACHE_DIR name an exact directory.
  • Watch. A rename no longer makes --watch or an opened root re-walk the whole tree; each side is verified as its own path. On a 476k-entry macOS tree where atomic temp-file writes rename constantly, a 620-second soak beside the old build reconciled the root 0 times rather than 24 and used 0.8% of a core rather than 33%. On macOS a file held open for writing, such as a growing log, now shows its size as of its last close, because the kernel reports those writes only then.

Install

uvx --no-build fdu@latest --help    # run the latest release without installing it
uv tool install --no-build fdu      # the command line from a prebuilt wheel
cargo install --locked fdu          # the command line, built with Rust 1.85 or newer
pip install fdu                     # the Python package: import fdu
cargo add fdu                       # the Rust library, re-exported with the command line
cargo add fdu-core --features watch # the engine alone, with the watch layer

uvx --no-build fdu@latest --install-skill writes the agent skill under the project root; re-run it after upgrading so the skill describes this release.

Platforms

Platform Command line and library Python wheel
macOS 11+, x86-64 and arm64 Supported; the scanner uses getattrlistbulk abi3 wheel
Linux glibc 2.17+, x86-64 and arm64 Supported abi3 wheel (manylinux2014); arm64 is cross-built and inspected, not run, before release
Windows x86-64 Supported, with no performance claims abi3 wheel

The Python package supports GIL-enabled CPython 3.12 and newer through one stable-ABI (abi3) extension. Other systems, including musl Linux and Windows arm64, build the source distribution with Rust 1.85 or newer.

Compatibility

fdu is 0.x: a minor release may change the command line or the Rust or Python API incompatibly, and the CHANGELOG names each change. This release emits fdu.report/10 for every report and fdu.cache/3 for cache status; the watch stream stays fdu.stream/2. Treat an unrecognized schema as a version you cannot parse rather than guessing at its fields.

Upgrading from 0.1.0

  • Flags. --cache only is --stale-ok, --cache refresh is --cache on, and --cache read-only is gone; each is refused with its replacement named. --exclude-ignored is --ignored=exclude, --only-ignored is --ignored=only, and --analysis-workers is --workers; these three are refused as unknown arguments, and the parser’s suggestion does not name the replacement.
  • A one-shot report no longer primes later runs. A default fdu PATH leaves no snapshot for a later open, --watch, or --analyze run of the same root, which therefore starts with a walk. Add --cache on to the report if a later --stale-ok read or a warm start needs one. The one-shot report itself never read the snapshot: revalidating one stats every entry anyway, so it cannot make a metadata report cheaper, and on Linux it measured slower than a fresh walk.
  • Machine output. Move a consumer pinned to fdu.report/7 to fdu.report/10, and one pinned to fdu.cache/2 to fdu.cache/3.
  • Cache files. 0.2.0 neither reads nor removes what 0.1.0 wrote, so each cached tree’s first run walks it. On macOS, delete the old ~/Library/Caches/fdu directory. Elsewhere the old <key>.fdu and <key>.content files sit beside the new ones, and fdu --cache-status=all lists them as unrecognized; delete them by hand, or run 0.1.0’s fdu --cache-clear=all before upgrading.
  • Rust. CachePolicy is Auto, On, Off; Plan::persists replaces CachePolicy::writes; and a Delivery built by struct literal must name stale_ok, or use Delivery::new or Delivery::stale_ok.
  • Python. CachePolicy.ONLY becomes stale_ok=True on fdu.report and fdu.open, CachePolicy.REFRESH becomes CachePolicy.ON, and CachePolicy.READ_ONLY is gone.

Known Limitations

  • The indexed tree on Linux is slower than pdu and diskus, as above.
  • Memory. Every report but an unfiltered --no-gitignore --view summary builds the exact index, so peak memory grows with the entries retained: 322 MiB for the million-entry indexed tree on Linux and 286 MiB on macOS, against 3.7 MiB for pdu on Linux. Freeing the index off the answer path returns the answer sooner; it does not lower the peak.
  • Performance evidence for this release is from a quiet virtualized Linux host and a busy bare-metal macOS host whose runs were declared uncontrolled, so a macOS change under about 3% of wall time is unresolved rather than absent. Windows has none.
  • Open writers on macOS. --watch and an opened root miss writes to a file that stays open until it closes; a one-shot report reads its current size.
  • The 0.1.0 limitations on links, cache retention, .gitignore fidelity, watch backends, opened roots, and JSON integers still apply; the CHANGELOG lists them.

Security

Report vulnerabilities through GitHub’s private vulnerability reporting, as SECURITY.md describes.

Full changelog: CHANGELOG.md

Full commit history: changes from v0.1.0 to v0.2.0