Skip to content

Releases

fdu 0.6.0

Choose a tag to compare

@jlevy jlevy released this 12 Nov 22:29
Immutable release. Only release title and notes can be modified.
4450be7

fdu 0.6.0 finds what was written recently, from one scan with no saved state, and shows how full the disk is beside what the tree holds. Every time in a report is read on one clock, change time by default, which catches copies and extracted archives that modification time misses.

This release renames the time flags and machine-output fields, changes the report schema to fdu.report/12 and the watch stream to fdu.stream/3, and changes parts of the Rust and Python APIs. fdu is alpha software, so the old spellings are gone rather than kept as aliases. See Upgrading from 0.5.0 before upgrading a script, a parser, or a library caller.

What’s Changed

Features

  • What was written recently. fdu PATH --kind file --since 24h shows where the data written in the last day now lives, by folder and largest first, and fdu PATH --view largest --since 24h lists the largest of those files. --since is inclusive and --before exclusive; each takes an age such as 2h or 30d, or an RFC 3339 instant. Keep --kind file: without it, a directory with any activity in the window counts at its full size. The answer is data written, not net growth: a file written in the window counts in full, and deleted files do not appear.

  • One clock for every time, change time by default. A report reads one clock for its window, --sort time, the tree’s age column, --long, recent, and ages, so it never mixes two. The default is change time: the newer of an entry’s modification time and its status change, which the operating system sets whenever data or metadata is written. It catches data that arrived carrying an older modification time, as cp -p, rsync -a, Finder and Explorer copies, mv from another filesystem, extracted archives, and Cargo’s crate sources do. Its errors point the safe way for both questions a time answers. Asked what is new, it can over-include, since a chmod looks new, rather than omit a copy. Asked what is old, it does not call a just-restored backup stale.

  • --time modified when change time says too much. It reads modification time for the whole report, as ls -l shows. Use it after a bulk chmod, chown, or relabel, and in a tree whose files are hard-linked from elsewhere. Making or removing any hard link moves the change time of every other name of the file, which uv and pnpm installs from a shared store, cp -l, and git clone --local do routinely: after uv 0.12.1 installed the same two packages into a second environment on ext4, 26 files (632 KiB) of the first read as newly written, and none did under --time modified. On Windows, a file fdu could list but not open has no observed status change; it is timed by its modification time, and a note counts such files.

  • --view ages: when the space arrived. The selected regular files are counted in nine fixed bands on the report’s clock, newest first, from future and under 1h to over 3y:

    $ fdu / --one-filesystem --view=ages
       7.7 GiB   18.9%  under 1h     9,004 files (7.7 GiB gitignored)
        25 GiB   61.0%  1h to 1d     93,255 files (18 GiB gitignored)
       3.8 GiB    9.4%  7d to 30d    61,112 files (768 MiB gitignored)
       984 MiB    2.3%  30d to 90d   12,810 files
       551 MiB    1.3%  90d to 1y    7,147 files
       2.0 GiB    5.1%  1y to 3y     54,225 files (66 MiB gitignored)
       848 MiB    2.0%  over 3y      37,542 files (24 KiB gitignored)

    The bands add up to --view summary for any selection. Where the selection admits files alone, each band is also what --kind file with the window it names selects. The view reads metadata only, shares one walk with other views (--view tree,ages --kind file --since 7d), and is part of --view full. Machine output keeps every band, with its edges as ages and as instants and a cumulative tally of it and every newer band.

  • --view volumes: how full the disk is. Each filesystem the report touches gets three lines: its capacity, share in use, path, and type; what is available; and what this report counted there.

    $ fdu ~/.cargo --view=volumes
       251 GiB   96% in use  .  (ext2/3/4)
                             10 GiB available
                             318 MiB counted by this report

    A walk without --one-filesystem shows each mount below the path as its own row, and fdu PATH --view volumes --scan-depth 0 reads capacity alone. Capacity is observed when the report is read and never cached, so --stale-ok shows current capacity beside a stale tree. Reading it mounts nothing the tree view would not: below the root, a filesystem is asked only where the walk listed it and, on Linux and macOS, the mount table shows a mounted filesystem, not an automount trigger, at exactly that path. The table is read once per report, so an automount that expires between that read and the call there can be mounted again, in a window as long as the calls before it. Anything else keeps its row and its counts with capacity unknown and the reason. Rows never add up: an APFS, btrfs, or ZFS row says its available space is shared within its pool, and snapshots and deleted files still held open use space no walk sees. The view was exercised on Linux, real automounts included. On macOS and Windows it is covered by automated tests, and its output on real APFS and NTFS volumes has not been checked by hand.

  • Selection coverage. A report whose selection narrows the scan ends with a note such as note: selected 2,951 of 27,060 files, 704 MiB of 1.8 GiB, so a filtered root’s 100% is not read as the whole tree. Machine output carries both as selected and scanned on every report.

  • Both clocks in machine output. request.time names the report’s clock, and every list row gives time_ns on that clock beside the raw mtime_ns and changed_ns. An arrival that kept old content times is a row whose changed_ns is recent and whose mtime_ns is old. Watch upserts carry changed_ns too.

  • Python. Selection(since=, before=, time=TimeBasis.MODIFIED), SortKey.TIME, View.AGES, and View.VOLUMES mirror the command line, with typed AgesSection and VolumesSection results and Report.selected and Report.scanned. Index.freshness reads a retained index’s freshness without building a report.

Fixes

  • Automount points on Linux. Under --one-filesystem, a kernel automount point that is not mounted yet, such as an NFS nested export or tracefs under debugfs, is now a boundary on glibc builds, which the wheels are. 0.5.0 listed it on a first run, which mounted the other filesystem and counted the files at its top, and stopped at it on the next. A directory the walk enters on another device than its parent, such as a direct automount trigger or a kernel automount point the walk itself mounts, is also recorded as the filesystem found there, so a first run and the next agree about it. A browsable key of an indirect autofs map is not yet: it is on its parent’s device until listed, so it is still recorded as the stat found it, and the two runs can disagree about it.
  • Python roll-ups. Index.total() and Index.rollup() take a roll-up and its provenance from one native read. They made two, so a refresh between them could pair a roll-up with the provenance of a later state.
  • Python opened reads. OpenedIndex.read raises TypeError for an argument that is not a projection, where 0.5.0 read it as Diagnostics.
  • Incomplete-subtree note. note: incomplete subtrees remain visible below the size threshold is printed only when the tree shows such a row. 0.5.0 printed it whenever a share threshold applied and some directory was not listed in full, even when the selection gave that directory no row, as --kind file does for an unreadable directory, or when --depth, --breadth, or --limit hid the row.

Guidelines and Content

  • Agent skill. The installed skill shows how to find where recently written data is using disk space, explains the change clock and when --time modified is the one to use, and covers ages, volumes, the coverage note, fdu.report/12, and fdu.stream/3. fdu --help and fdu --docs describe the same commands. The skill’s recipe for recent writes is one walk, fdu PATH --kind file --since 24h --view tree,summary,largest, which gives the folders, their total, and the largest files. To ask more of the same tree without walking it again, add --cache on to that command and --stale-ok to each later one, which answers from the first walk’s snapshot and labels the answer stale. Give every command the same --cache-dir, outside the scanned path, since a snapshot inside it is itself a file written in the window. Re-run fdu --install-skill after upgrading, since a skill names the version that generated it.

Documentation

Upgrading from 0.5.0

  • Flags: --modified-since and --modified-before are --since and --before, and --sort mtime is --sort time. The old spellings are refused: each exits 2 with one line naming its replacement and the clock that changed with it, such as fdu: --modified-since was renamed --since in 0.6.0, and the window reads change time unless --time=modified is given. --sort time with only types, extensions, families, languages, or documents is refused as well, since their rows carry no time; 0.5.0 accepted --sort mtime there and ordered them by name. Beside a view that has a time, they keep size order and a note...
Read more

fdu 0.5.0

Choose a tag to compare

@jlevy jlevy released this 14 Oct 01:20
Immutable release. Only release title and notes can be modified.
4a513b0

fdu 0.5.0 shows how recently each part of a tree changed and combines several disjoint paths into one report. Tree sizes and ages use colors to make large and recent entries easier to find.

This release changes the report schema to fdu.report/11, tree sorting by modification time, and parts of the Rust and Python APIs. See Upgrading from 0.4.0 before upgrading a parser or library caller.

fdu 0.5.0 on the Linux kernel source: tree sizes and ages, then code and document counts

The usage tree, then code and document counts, cold and cached, on the Linux kernel source (1.7 GiB, 95,938 files), recorded in real time on an M1 Pro. These are individual runs, not benchmarks.

What’s Changed

Features

  • Tree ages and colors. Each row shows the age of its newest counted entry, including directories and symlinks. Filters determine which entries count; the scan root’s own modification time does not. Incomplete rows show unknown, empty rows show —, and the remainder row has no age. Colors distinguish recent activity and size ranges. Other report colors and flat-list formats keep their existing styles.
  • Several roots in one report. fdu docs src shows a combined total with each root beneath it; fdu docs src --view largest --limit 10 finds the ten largest files across both. Counts, shares, sorting, and bounds apply to the combined report. Every root must be a directory, and repeated or overlapping roots are refused. Python’s fdu.report accepts a sequence of paths as well as one path.
  • Machine-readable modification times. Tree nodes expose exact modification and age nanoseconds and completeness. Tree and list rows include RFC 3339 UTC modified_at, with age_reference_at on the report. Python exposes the rendered times as timezone-aware UTC datetime values, floored to microseconds.
  • Ages during a watch. A repaint measures ages at the time it paints. Directory ages reflect entries created, removed, or renamed inside them, as a one-shot scan does. The query’s time bounds remain the bounds resolved when it was created.

Guidelines and Content

  • Agent skill. The installed skill explains multi-root reports, combined bounds, root indexes in machine output, tree ages, and fdu.report/11. It distinguishes fdu */ from a single walk of the parent and explains their cost and symlink differences.

Documentation

  • Performance evidence. Linux experiments record the cost of the age column and multiple roots. The current evidence does not establish a lead over pdu --max-depth 2 or diskus on the two real trees; older tables are labeled as historical measurements. The dense-tree one-root summary result is inconclusive at the registered +3% non-inferiority margin. See Performance Measurements for the measurements and their limits.

Upgrading from 0.4.0

  • Machine output: accept fdu.report/11 explicitly. A multi-root report has root: null and a roots list. Rows, errors, and refusals identify their root by index. Its tree section has tree: null, a total, and a trees list, with a remainder for each root. Single-root reports keep their existing root shape and gain the new time fields. The stream and cache schemas remain fdu.stream/2 and fdu.cache/3. See the machine-output reference.
  • Text output: the tree has an age column. --sort mtime orders tree rows by the newest counted activity of any kind, rather than by regular files alone. newest_mtime_ns retains its regular-files-only meaning. Ages beyond a month use months of 30.44 days and years of 365.25 days, in the tree and --long. Time filters still take days: use --modified-before 60d, not 2mo.
  • Rust: Report::root becomes Option<PathBuf>, alongside Report::roots. TreeNode, FileRow, RefusedControl, ProgressSnapshot, and Delivery gain fields; Section::Tree gains roots; tree status errors use StatusIssue. Update struct literals and exhaustive matches. Roots, NamedRoot, and the *_roots* entry points provide multi-root requests and execution.
  • Python: Report.root is Path | None; use Report.roots for multi-root reports and TreeSection.total and TreeSection.trees for their tree sections. File rows, errors, and refusals carry their root index.
  • Commands taking one root: --watch, --cache-status, and --cache-clear continue to require a single path. A file supplied as a root is rejected as a non-directory before its snapshot is read.
  • Many small roots: each root pays a walk’s fixed cost. Use fdu --depth 1 --min-share 0% . when the intended question is the size of everything one level below a shared parent. Unlike fdu */, it also includes hidden entries and files and does not follow a symlink named as a root.

The changelog lists the API changes in detail.

Full commit history: v0.4.0 … v0.5.0

fdu 0.4.0

Choose a tag to compare

@jlevy jlevy released this 09 Oct 07:26
Immutable release. Only release title and notes can be modified.
c041ed1

fdu 0.4.0 changes what a report asks for and what it says. A content view now runs the analysis it shows: fdu . --view code,documents counts lines of code by language and words by document type in one scan, with no --analyze, where 0.3.0 refused a content view until you named its analyzer. A human report now prints only the result on standard output, and every explanation follows it on standard error as a short note: or tip: line, consolidated so that one note says what the totals include, one lists every display limit that hid something, and one runnable tip lifts them all. documents percentages now add up to 100%.

The release is breaking in a few places, listed in Upgrading from 0.3.0: --view=documents and --view=full beside --analyze=lines or --analyze=code, and four Rust API types. No option was removed, and the report, cache, and stream schemas are unchanged.

fdu 0.4.0 on the Linux kernel source: the usage tree, then lines of code by language and words by document type, first with fdu’s cache cleared and then from the cache

fdu . and then fdu . --view code,documents, cold and cached, on the Linux kernel source (1.7 GiB, 95,938 files), recorded in real time on an M1 Pro: one run of each command, not a benchmark. The full-resolution video, fdu-demo.mp4, is attached to this release.

What’s Changed

Features

  • Content views run the analysis they show. --view=code runs code analysis and --view=documents runs words analysis, so fdu . --view code,documents gives both rundowns from one scan. --analyze now adds analyzers beyond what the views imply: analysis shown in a metadata view (--analyze=code --view=languages), analysis run only to warm the content cache, or a wider set. Views with a metadata meaning (tree, languages, types, families, full) never imply analysis. In Python, fdu.report(root, Query(views=(View.CODE,))) runs code analysis the same way. An index from fdu.open or fdu.scan keeps exactly the analyzers it was opened with, and a view it cannot answer is refused with the analyzer named. In machine output, request.analyze lists every analyzer that ran, named or implied.

  • Notes and tips after the result. Human reports hold rows, column headings, and multi-view section headers; everything that explains them is a note: or tip: line on standard error after the result. That includes what used to sit inside the result: the percentage denominator (Percentage column: code lines is now note: percentages are shares of code lines), the row bound, share-floor omissions, Ranked by, and the code overview’s coverage. The notes are consolidated:

    • one note for what totals include;
    • one note listing every display limit that hid something, such as note: display limits: below 1% of root, depth 5;
    • one runnable tip that lifts them all, such as tip: show more: --min-share=0% --depth=all (in Python, min_share=0%, depth=all).

    Beside other views, a note names the section it is about: note: percentages are shares of code lines (CODE), document words (DOCUMENTS). --quiet still suppresses notes and tips. Grouped rows are narrower too: a row’s first line ends at its file count, and each further measure, such as lines, words and pages, or generated files, is a line of its own indented under it, so the report fdu . --view code,documents prints on the Linux kernel source fits in 88 columns, where 0.3.0’s widest DOCUMENTS row took 154 (the perf: line on standard error is still longer). Those rows no longer show 0.3.0’s documentation count (4,063 documentation), which counts files in a doc, docs, or documentation directory or named like a README, CHANGELOG, or CONTRIBUTING file: in DOCUMENTS that is nearly every file, and it read as a measure of the documents. JSON, JSONL, and YAML keep it.

  • Refusals name their remedy. A message about analysis names the change that fixes it and a route that accepts it: note: code analysis not shown by summary with tip: show it: --view code, or note: full omits code, documents without analysis with tip: include them: --analyze all. A refused read of an index names what it was opened with, and an opened root says it runs no analyzer.

Fixes

  • documents percentages add up to 100%. Each row’s share divided its document words by the total row’s, but logical words are derived after pooling, so a pooled total of mixed formats differed from the sum of its rows: with a long-token format such as HTML beside ordinary prose, 0.3.0’s shares summed to 101.6% on the Linux kernel source, and with very short tokens they could sum well below 100%. Shares in every grouped view now divide by the sum of the rows’ values, which is what share.denominator and the total row’s share carry in machine output. Byte, code-line, and raw-word shares are unchanged, since their rows already summed to their totals.

Guidelines and Content

  • Agent skill. The skill that --install-skill writes leads with the tree, the code and documents rundown, and JSON output; tells an agent to read the note: and tip: lines after a result; and quotes the section-named percentage note and the tip for a partly shown report.

Documentation

  • README. The homepage is restructured: it opens with the recording above, leads with the tree, then the code and documents rundown, and its Speed section says why fdu is fast and labels its macOS comparison as exploratory.

Install

uvx --no-build fdu@latest --help    # run the latest release without installing it
uv tool install --no-build fdu      # the command line from a prebuilt wheel
cargo install --locked fdu          # the command line, built with Rust 1.85 or newer
pip install fdu                     # the Python package: import fdu
cargo add fdu                       # the Rust library, re-exported with the command line
cargo add fdu-core --features watch # the engine alone, with the watch layer

Re-run uvx --no-build fdu@latest --install-skill after upgrading so the agent skill describes this release.

Compatibility

fdu is 0.x: a minor release may change the command line or the Rust or Python API incompatibly, and the CHANGELOG names each change. Reports stay fdu.report/10, cache status fdu.cache/3, and the watch stream fdu.stream/2. No command-line option was removed.

Upgrading from 0.3.0

  • First run scans cold. 0.4.0 reuses no snapshot or content sidecar written by 0.3.0, since the engine fingerprint includes the release version: each cached tree’s first run walks it, and its first analyzed run reads every file again.
  • --view=documents runs words analysis. Beside --analyze=lines or --analyze=code it now runs words analysis as well, because documents implies it. --analyze=lines --view=documents --format=json reports request.analyze: ["lines", "words"] and share_metric: "document_words" where 0.3.0 reported ["lines"] and raw_words; a script that read raw-word shares by document type reads document_words instead. On a held index with lines analysis only, --view=documents is refused.
  • --view=full without words analysis omits DOCUMENTS. Under --analyze=lines or --analyze=code, the report lists DOCUMENTS in omitted_views and says note: full omits documents without words analysis with tip: include them: --analyze all. A script that read that section adds words to --analyze.
  • Human output. Lines that explained a result, such as (2 of 15), Percentage column: …, and Ranked by …, are now note: lines on standard error, and a grouped row puts each measure after its file count on a line of its own and no longer shows the documentation count. A script that parsed them from standard output reads the machine formats instead, where a row’s detection.flags.documentation still holds that count.
  • Rust: query::Request. It gains a private field recording which analyzers the caller named and which views implied the rest. Outside the engine crate, build one with Request::new, Request::read, or Request::build rather than a struct literal; its public fields basis, query, and now are unchanged.
  • Rust: query::RequestError. It is now #[non_exhaustive]; an exhaustive match adds a wildcard arm. Its analysis refusals change shape: ViewNeedsContent(ViewSpec) and NeedsAnalyzer { item, analyzer } are replaced by ViewNeedsAnalyzer { view, held, holder } and SortNeedsAnalyzer { metric, analyzer, held, holder }, where holder is the new BasisHolder; WatchContent becomes WatchContent { named, views }; and AnalyzerNamedAsView::suggested_view is a ViewSpec (its label() is the old string). Each of these variants is #[non_exhaustive], so match them with ... Code that renders a refusal with message or Display is unaffected.
  • Rust: ViewSpec::full_report. Under AnalysisSet::LINES_ONLY or CODE_ONLY it returns Documents among the omitted views rather than the shown ones.
  • Rust: query::AxisNames. It gains a public field, setting_separator, which joins several settings in one suggestion.
  • Python. fdu.report with View.CODE or View.DOCUMENTS runs that analysis instead of refusing. index.report on an index opened without the analyzer raises InvalidArgumentError naming it; open with AnalysisOptions(analyze="code") or "words" to serve those views.

Known Limitations

  • DOCUMENTS totals are pooled. The total row’s document_words and pages are counted from statistics pooled across all files, so on a tree of mixed formats the rows can sum to more or less than the total row, while the percentages divide by the rows’ sum.
  • **A watch’s remedy can point at a re...
Read more

fdu 0.3.0

Choose a tag to compare

@github-actions github-actions released this 30 Sep 17:26
Immutable release. Only release title and notes can be modified.
17b23c1

fdu 0.3.0 makes the default command faster on Linux and fixes how a scan treats automount points, unusual .gitignore files, and very large files under content analysis. On Linux the default fdu PATH tree took 48% less time than 0.2.1’s on the Linux kernel source, 14% less on a node_modules tree, and 15% less on a generated million-entry tree, where its peak memory fell from 293 to 57 MiB. In one paired run per tree on a virtualized Linux host, fdu’s default command led pdu, at its default and at --max-depth 2, and diskus on all three trees. A scan on Linux no longer mounts an unmounted automount point by statting it, a .gitignore with a byte-order mark or a NUL byte reads as git reads it, and content analysis no longer holds a one-line source or a large Markdown file whole in memory.

Four Rust API changes are incompatible: counters::Counts, scan::ScanBackendDiagnostics, Error, and content::CoverageReason each gain fields or a variant, and Counts and ScanBackendDiagnostics are now #[non_exhaustive]. No command-line option, report or cache schema, or Python API changed. Read Upgrading from 0.2.1 before updating a Rust caller.

Performance

Every figure here is from Linux, on a quiet 4-vCPU virtualized ext4 host with warm filesystem caches, in paired, interleaved runs. The real trees are the Linux v6.12 source (92,474 entries, 358 .gitignore files) and a directory-dense node_modules (79,957 entries); the generated tree has 1,000,001 entries.

  • Against 0.2.1. Measured end to end in one cell of 20 pairs per tree, the release engine’s default tree took 48% less time than 0.2.1’s on the kernel source (209 to 110 ms), 14% less on node_modules (135 to 116 ms), and 15% less on the generated tree (1.11 to 0.95 s), and its default summary 35%, 12%, and 17% less (exp-202). The development records measured the same work in steps: the overnight round took the default tree 39% and 10% off and the default summary 26% and 7% (exp-194, exp-195), and three later changes took the default tree another 3.1% and 8.9% off and the default summary 6.1% on the kernel source (exp-201).
  • A default tree that keeps only the files it can show. When a one-shot report needs nothing but the tree, as the default fdu PATH does, fdu keeps every directory and only the largest files that could reach the smallest share the tree prints, and folds the rest into their directories’ totals. The answer is unchanged. On the million-entry tree the default tree’s peak memory fell 80%, from 293 to 57 MiB (exp-202). The tree also decides from each child’s totals whether the child will show before building its row, and on Linux takes each directory’s and symlink’s kind from its parent’s listing rather than statting it again (exp-197, exp-199).
  • Faster .gitignore classification. Each ignore file’s rules are indexed by literal name, extension, and suffix, and the rules left over are ruled out by byte checks before any glob runs. On the kernel source, what .gitignore handling adds to the default tree fell from 52% of a walk without it to 1.6% (exp-178, exp-194). The default summary finds the rules for each directory by its bytes rather than its path components (exp-198). Which paths the rules match is unchanged, and fdu’s ignored set agrees with git check-ignore on every path of the kernel source.
  • A native directory reader on Linux. Builds against glibc, which include the Linux Python wheels, read each directory with getdents64 into a reused buffer and stat its entries with statx relative to the directory, on every route that lists directories. A directory the reader cannot list is read the portable way, and musl builds use only the portable reader. On the kernel source the default command’s fstat calls fell from 5,773 to 4, and the default summary took 9–10% less time on both real trees (exp-185, exp-186). Its tests run on x86-64 and arm64 Linux.

Against its peers, each paired 20 times with an adjacent fdu run in one interleaved run per tree, the release’s default command led on all three trees (exp-202):

Tool Work returned Linux v6.12 source node_modules Generated, 1M entries
fdu default tree: five levels, 1% share floor 0.110 s, baseline 0.106 s, baseline 0.951 s, baseline
pdu 0.24.0 --max-depth 2 the root and its children +10% [+2%, +12%] +12% [+2%, +19%] +19% [+15%, +20%]
pdu 0.24.0 default tree: ten levels, 1% floor +15% [+12%, +19%] +18% [+15%, +23%] +25% [+21%, +28%]
diskus 0.9.0 one total +17% [+3%, +20%] +12% [+9%, +17%] +24% [+20%, +28%]
fdu 0.2.1 the same default tree +95% [+86%, +112%] +20% [+18%, +22%] +21% [+18%, +22%]

Each percentage is the median paired change in wall time against the adjacent fdu run, with its 95% interval; positive means the peer took longer. fdu’s time includes reading the kernel tree’s .gitignore files, which neither peer reads, and on every tree its output matched 0.2.1’s byte for byte. The narrowest leads are over pdu --max-depth 2: on both real trees their intervals reach down to +2%. How large a lead is depends on the session. This one ran about 30% slower than exp-201’s for every tool on the real trees; in exp-201 the leads over pdu --max-depth 2 were 3% and 10%. On the generated tree 0.2.1 was level with pdu --max-depth 2 in this run, where earlier sessions’ figures put pdu about 11% ahead of it, so by that estimate about ten points of the 19% lead reflect this session’s host. The previous run on that tree, on an earlier engine from this cycle, had pdu --max-depth 2 3% faster than fdu’s default tree (Linux comparison).

None of these changes was measured on macOS or Windows. The native reader and the automount fix are Linux-only, and taking kinds from the listing changes nothing on macOS, whose listing already carries every child’s attributes, or on Windows, which stats every entry; the folded tree and the .gitignore changes apply on every platform.

What’s Changed

Features

  • .gitignore counters. FDU_COUNTERS=1 reports three more rows in its control state group: ignore patterns tested, the rules whose glob ran against an entry; ignore bucket probes, the lookups of an entry’s name in an ignore file’s indexed rules; and ignore bucket hits, the lookups that found a matching rule.
  • Watch entry points in Rust. watch_session::Session::changed_report answers only when what a reader would see has changed, and changed_report_with_progress does the same while reporting its build as Summarizing; fdu --watch uses both.

Fixes

  • Automount points on Linux. No stat of a listed child triggers an automount: an unmounted autofs trigger under the scanned tree (/net, /misc, a systemd automount unit) is reported as the trigger, as lstat and GNU du report it, and a walk no longer mounts, or hangs on, a network filesystem it would not descend into. 0.2.1 mounted it on glibc builds; musl builds were never affected (exp-196 screened the change for regressions).
  • Directories that list but cannot be searched. A directory that is readable but not searchable (mode 0400) is reported the same way on every route: each of its children as an error, and none as an entry. In 0.2.1 the default --view summary counted such a directory’s subdirectory as a directory and reported it again when it failed to open, and reported nothing for its symlink.
  • .gitignore byte-order marks and NUL bytes. A .gitignore that starts with a UTF-8 byte-order mark now applies its first rule, and a pattern ends at the first NUL byte in its line, as git reads both. 0.2.1 matched the mark as part of the first rule, so that rule never applied, and kept the bytes after a NUL as part of the rule.
  • Content analysis memory. Each worker now holds a bounded part of a file:
    • --analyze code classifies a line in pieces over a 64 KiB window as it arrives, so a one-line minified or generated source costs a worker the window, not the file. Every count is unchanged.
    • --analyze words renders a Markdown file of u...
Read more

fdu 0.2.1

Choose a tag to compare

@jlevy jlevy released this 29 Sep 05:49
Immutable release. Only release title and notes can be modified.
c164457

fdu 0.2.1 is a patch release focused on Linux. The default fdu PATH and --view summary are about three times faster on source trees with many .gitignore files, the default summary no longer builds a full index to report its ignored share, and building the full index is faster on trees of many small directories. A .gitignore spelled in another case now applies where git applies it, and an answer served from the cache with --stale-ok always says it is stale. There are no incompatible changes to the command line, the report schemas, or the Rust and Python APIs.

Performance

Three engine changes were each measured on their own, on a quiet 4-vCPU virtualized ext4 host with warm caches:

  • Ignore rules without allocation, resolved once per directory. On the Linux v6.12 source tree (92,474 entries, 358 .gitignore files) the default tree fell from 590 to 211 ms and the default summary from 505 to 167 ms, with identical answers (exp-173, exp-174).
  • A summary that reads ignore rules without an index. On a million-entry tree with no .gitignore the default summary took 19% less time and its peak memory fell from 314 to 8.6 MiB; on the source tree, 7% less time and 23% less memory (exp-187). On macOS its peak memory fell 58–69%.
  • Directory listings freed where they were allocated. The default indexed tree took 8.6% less time on a node_modules tree of small directories, and a screen on the generated million-entry tree measured 10.6% less; building the index alone took about 7% less on both. Trees with few, larger directories are unchanged (exp-190).

Against its peers on the generated million-entry Linux tree, 0.2.0’s summary (--no-gitignore --view summary) was already the fastest tool measured, 8% ahead of pdu and 12% ahead of diskus, while its indexed tree took 23% longer than pdu. The listing change above cuts that job by about a tenth; the peer comparison has not been re-run on 0.2.1. The pdu brief explains where the remaining Linux gaps come from and what comes next. On macOS, where fdu was already the fastest tool measured, these changes were not re-measured for this release.

Highlights

  • Case-variant .gitignore. On a case-insensitive volume (APFS and NTFS by default, an ext4 casefold directory), a .gitignore spelled .GITIGNORE or in any other case now governs, as it does for git; on a case-sensitive volume it stays an ordinary file. Every route agrees, and the rules are reported under .gitignore.
  • Stale answers say so. A --stale-ok answer prints warn: stale answer: served from the snapshot without filesystem verification on stderr on every format, and --quiet keeps it. Python reports carry it in the new Report.warnings.

Install

uvx --no-build fdu@latest --help    # run the latest release without installing it
uv tool install --no-build fdu      # the command line from a prebuilt wheel
cargo install --locked fdu          # the command line, built with Rust 1.85 or newer
pip install fdu                     # the Python package: import fdu

Re-run uvx --no-build fdu@latest --install-skill after upgrading so the agent skill describes this release.

Compatibility

This is a patch release: no command-line option, report or cache schema, or Rust or Python API changed incompatibly, and cargo-semver-checks finds no breaking change in fdu-core or fdu against 0.2.0. Two additions: Python Report.warnings and Rust report_format::report_warnings.

Upgrading from 0.2.0

  • First run scans cold. As with every release, the engine fingerprint changes, so 0.2.1 reuses no snapshot or content sidecar written by 0.2.0: each cached tree’s first run walks it, and its first analyzed run reads every file again.
  • Stderr. A script that expects empty stderr from a --stale-ok run now sees one warn: line; paths and long output no longer print note: cache-only result.

Known Limitations

  • Linux. On source trees with many .gitignore files the default command still takes about three times pdu’s time, because classification runs on one thread. Building the full index on the generated tree took 23% longer than pdu in 0.2.0; by the listing change’s measured saving that gap is about 10%, not yet re-measured against peers.
  • Memory on Linux source trees. The ignore-aware summary holds about 20 MiB more than a walk without .gitignore on the Linux v6.12 tree, which is not yet explained.
  • The 0.2.0 limitations on links, cache retention, watch backends, opened roots, and JSON integers still apply.

Security

Publishing now refuses a release tag that is lightweight, off main, or signed with a signature GitHub cannot verify, checks the Rust API with cargo-semver-checks before uploading, and publishes to crates.io only through trusted publishing. Report vulnerabilities through GitHub’s private vulnerability reporting, as SECURITY.md describes.

Full changelog: CHANGELOG.md

Full commit history: changes from v0.2.0 to v0.2.1

fdu 0.2.0

Choose a tag to compare

@jlevy jlevy released this 28 Sep 17:56
Immutable release. Only release title and notes can be modified.
v0.2.0
6ec7716

fdu 0.2.0 makes the command people type cheaper. The default fdu PATH report no longer writes a cache snapshot that no later report reads, a large one-shot index is freed after the answer instead of before it, and a report that asks for several content views resolves each file once. The cache contract is simpler: --cache takes auto, on, or off, and answering from a snapshot without touching the tree is its own flag, --stale-ok. Code analysis gains an aligned code table and population control, and tree output gains display bounds. The same engine still ships three ways: the fdu command line, the fdu Python package, and the fdu-core Rust crate.

This release changes flags, both versioned report and cache-status documents, the Rust and Python cache policy, and the macOS cache location. Read Upgrading from 0.1.0 before updating a script, a machine-output consumer, or a library caller.

Performance

On Linux, fdu was measured against seven other disk-usage tools for the first time, on the same generated 1,000,001-entry tree and paired harness as the macOS comparison, on a quiet 4-vCPU virtualized ext4 host. The two fdu jobs rank differently there:

  • fdu --no-gitignore --view summary is the fastest tool measured, at a 0.94-second median. pdu took 8% longer and diskus 12% longer.
  • The indexed tree is not. Building the reusable exact index and a ten-row tree with the cache off took 1.25 seconds, about 23% longer than pdu and 21% longer than diskus; every other peer was slower. A screen attributes the gap to allocator contention between fdu’s walker threads and its index builder rather than to filesystem work, and that hypothesis is still open.

The Linux comparison has the full matrix, and the cache economics brief says when the cache and the index pay on each platform.

On macOS the comparison was rerun on this release’s engine, on a bare-metal Apple M1 Pro with the same generated tree on internal APFS. fdu built the indexed tree in a 6.4-second median and was faster in paired wall time than every tool measured: dumac, which returns only a total, took 9% longer, pdu 49%, dust 57%, and BSD du about nine times as long. The host was busy, so the quiet gate declared the run uncontrolled; pairing keeps the relative figures, not the absolute seconds. The macOS comparison has the full matrix and the host regime.

Three engine changes in this release were each measured on their own:

  • No unread snapshot. Under --cache auto a one-shot metadata report no longer writes a snapshot. On the million-entry Linux tree the default tree job took 13.8% less wall time on a repeated run (95% interval 10.7% to 16.0% less) and 32.7% less on a first run into an empty cache directory, while a cold-scan placebo stayed within noise (exp-163). On macOS it cut the default tree job’s peak memory 26%, and its 3.1% wall saving was within that host’s noise (exp-165).
  • Index release off the answer path. A one-shot report frees an index of at least 65,536 entries on a detached thread after the answer is complete. Freeing the million-entry index had been 95 ms of a 1.39-second --cache off report, and the default tree job took 3.19% less wall time on Linux (95% interval 1.79% to 4.88% less, exp-160). On macOS, where the kernel dominates each run, the change neither helped nor hurt measurably (exp-164).
  • Multi-view reports resolve each file once. With two or more unfiltered metric views, each regular file’s content record and classification is resolved once for all of them, with output unchanged. On an exploratory, uncontrolled macOS host, 100 four-view report constructions over a 137,085-entry tree ran about 2.5 times faster (exp-159). This is not a scan or end-to-end speedup, and the change is retained provisionally until a quiet rerun settles its inconclusive major-fault gate.

Highlights

  • Cache policy. --cache auto, the default, reads and writes where the kind of run gains from it: content analysis, --watch, and an opened index, not a one-shot metadata report. --cache on also writes after a one-shot report, and --cache off neither reads nor writes. --stale-ok answers from the snapshot alone, says the answer is stale, and fails rather than scanning when there is no snapshot; it replaces --cache only. In Python these are CachePolicy.AUTO, ON, OFF and stale_ok=True; in Rust, CachePolicy::{Auto, On, Off} and Delivery::stale_ok.
  • Code analysis. --analyze code prints one aligned table of code lines, shares, comments, blanks, and analyzed-file coverage, per language and with a bold total that display limits never shrink. Multiline literals, heredocs, Rust lifetimes, and JavaScript regular expressions are counted correctly, which invalidates earlier code-analysis records. --workers sets content-analysis concurrency.
  • Population. --ignored=include|exclude|only chooses which side of .gitignore a report covers. Exclusion prunes safely ignored subtrees and skips their content analysis.
  • Tree display. The default tree goes five levels deep and hides rows under 1% of the selected root. --breadth bounds children per directory, --min-share sets the floor, --full removes every display bound, and a displayed directory always stands for its whole subtree, so the root’s rows and its remainder row add up to the root.
  • Cache location. Snapshots are <key>.metadata.bin with an optional <key>.analysis.bin, under ~/.cache/fdu on macOS and Linux and %LOCALAPPDATA%\fdu on Windows; --cache-dir and FDU_CACHE_DIR name an exact directory.
  • Watch. A rename no longer makes --watch or an opened root re-walk the whole tree; each side is verified as its own path. On a 476k-entry macOS tree where atomic temp-file writes rename constantly, a 620-second soak beside the old build reconciled the root 0 times rather than 24 and used 0.8% of a core rather than 33%. On macOS a file held open for writing, such as a growing log, now shows its size as of its last close, because the kernel reports those writes only then.

Install

uvx --no-build fdu@latest --help    # run the latest release without installing it
uv tool install --no-build fdu      # the command line from a prebuilt wheel
cargo install --locked fdu          # the command line, built with Rust 1.85 or newer
pip install fdu                     # the Python package: import fdu
cargo add fdu                       # the Rust library, re-exported with the command line
cargo add fdu-core --features watch # the engine alone, with the watch layer

uvx --no-build fdu@latest --install-skill writes the agent skill under the project root; re-run it after upgrading so the skill describes this release.

Platforms

Platform Command line and library Python wheel
macOS 11+, x86-64 and arm64 Supported; the scanner uses getattrlistbulk abi3 wheel
Linux glibc 2.17+, x86-64 and arm64 Supported abi3 wheel (manylinux2014); arm64 is cross-built and inspected, not run, before release
Windows x86-64 Supported, with no performance claims abi3 wheel

The Python package supports GIL-enabled CPython 3.12 and newer through one stable-ABI (abi3) extension. Other systems, including musl Linux and Windows arm64, build the source distribution with Rust 1.85 or newer.

Compatibility

fdu is 0.x: a minor release may change the command line or the Rust or Python API incompatibly, and the CHANGELOG names each change. This release emits fdu.report/10 for every report and fdu.cache/3 for cache status; the watch stream stays fdu.stream/2. Treat an unrecognized schema as a version you cannot parse rather than guessing at its fields.

Upgrading from 0.1.0

  • Flags. --cache only is --stale-ok, --cache refresh is --cache on, and --cache read-only is gone; each is refused with its replacement named. --exclude-ignored is --ignored=exclude, --only-ignored is --ignored=only, and --analysis-workers is --workers; these three are refused as unknown arguments, and the parser’s suggestion does not name the replacement.
  • A one-shot report no longer primes later runs. A default fdu PATH leaves no snapshot for a later open, --watch, or --analyze run of the same root, which therefore starts with a walk. Add --cache on to the report if a later --stale-ok read or a warm start needs one. The one-shot report itself never read the snapshot: revalidating one stats every entry anyway, so it cannot make a metadata report cheaper, and on Linux it measured slower than a fresh walk.
  • Machine output. Move a consumer pinned to fdu.report/7 to fdu.report/10, and one pinned to fdu.cache/2 to fdu.cache/3.
  • Cache files. 0.2.0 neither reads nor removes what 0.1.0 wrote, so each cached tree’s first run walks it. On macOS, delete the old ~/Library/Caches/fdu directory. Elsewhe...
Read more

fdu 0.1.0

Choose a tag to compare

@jlevy jlevy released this 26 Sep 05:10
Immutable release. Only release title and notes can be modified.
v0.1.0
7cf7f1b

fdu is a fast, incremental file roll-up engine. One walk over a directory tree answers, for every directory at once, how big it is, how many files it holds, what changed most recently, and what kinds of files it contains; the index is cached between runs and can be kept live as the tree changes. Reports form a cost ladder, from the aggregate-only fdu --no-gitignore --view summary PATH through the default fdu PATH tree and the path-only --view types to --analyze, the only option that reads file contents. This first release ships the same engine three ways: the fdu command line, the fdu Python package, and the fdu-core Rust crate.

On 2026-09-16, an exploratory macOS/APFS calibration of the release candidate built a reusable exact index and ten-row tree over a reproducible generated corpus of 1,000,001 entries in a 5.206-second median. The M1 Pro host ran under uncontrolled load, so this is not a portable absolute time or a cross-tool ranking; the full comparison preserves the complete evidence and its limits.

Highlights

  • Command line. fdu PATH prints a size-sorted tree, in allocated bytes unless --size apparent; --view picks among list (the default without --analyze), summary, tree, families, types, extensions, languages, documents, largest, recent, files, and full, and several views share one walk. Output is text, tree, paths, long, json, jsonl, or yaml, and exit status separates a complete result (0), a failure (1), and a partial result or usage error (2). At an interactive terminal, a run that takes longer than half a second shows one progress line on stderr, with its phase, the counts so far, and elapsed time, and erases it before any output; nothing is drawn into a pipe, a file, or CI, and --progress controls it. fdu --docs prints the usage guide, fdu --install-skill writes a portable skill for coding agents under the project root, and fdu --skill prints it.
  • Directory queries. Directories are selected like files, and each one’s size and age cover its whole eligible subtree, so fdu ~/projects --kind dir --include node_modules --modified-before 30d --long lists stale build directories with the bytes below each one and the age of its newest modification. --format paths prints only the paths, a flat list is complete and size-ranked, and --tree and --long are shorthands for --format tree and --format long. In JSON, JSON Lines, and YAML, a report with no --view or --analyze is this list view: one row per matching entry, with exact sizes, subtree counts, completeness, and a signed age against one report instant. --view tree gives the directory hierarchy.
  • Content analysis, only when asked: --analyze lines, code (standard lines of code for 15 languages), words (prose and reader-visible Markdown), or all, with results kept in a sidecar so unchanged files are not read again.
  • Python package. fdu.report, fdu.open, and fdu.scan return typed, immutable values, Report.as_dict() is the command line’s JSON, and Index.watch() is a closable change feed. Under the default auto cache policy, Index.refresh() and an iterated Index.watch() save what they verify, so a later cache-only open sees it. The wheel also installs the native fdu command.
  • Rust library. fdu-core is the engine, with open, a retained Index of pre-computed per-directory roll-ups, one-shot prepare_report, and an exact change feed; prepare_report_with_progress reports the phases the command line’s progress line draws. The fdu crate re-exports it with the command line.
  • Opened roots for interactive clients. OpenedIndex, in Rust and as fdu.opened in Python, answers while discovery is still running: coherent reads of several projections at one engine version, bounded pages with continuations, a resumable change journal, explicit refresh, discovery priorities, and a joined close.
  • .gitignore roll-ups, read by default on every surface: the command line, the one-shot library report, fdu.report, watch, and opened roots. Ignored and unignored totals for every directory come from every .gitignore inside the root, with git’s pattern semantics. Summary, tree, and extension rows end with the ignored part of their size, as (128 B ignored), and carry it as a tally in machine formats, where file rows carry a flag; --exclude-ignored and --only-ignored report one side, with sizes, ordering, and --min-size following the entries shown; and --no-gitignore, or read_controls=False, reads no rules and answers a typed “not observed” rather than guessing. Two limits bound the rules an index retains, --gitignore-budget SIZE|all (4 MiB by default) and --gitignore-line-limit SIZE|all (16 KiB): a .gitignore past either is refused rather than fatal, sizes stay exact, and a note names the directories skipped and the flag that lifts the limit that fired.
  • Cache and watch. A snapshot per root makes later runs cheaper where the cache supplies more than the filesystem will: --cache sets the policy, and --cache-status and --cache-clear manage snapshots without scanning. Status gives every file in the cache directory a state, and clearing reclaims snapshots another fdu version or format wrote, along with fdu’s own leftovers, while never removing a file that is not fdu’s. fdu --watch repeats the query as the tree changes, with every event verified by stat and every backend overflow reconciled. Cache-only content restore avoids repeated roll-up, candidate, classification, and path work, streams records into the retained index, and repeated unfiltered metric views share one entry traversal. Individual development comparisons on exploratory, uncontrolled, warm-steady M1 Pro/APFS runs measured roughly 4% to 13% lower wall time for cache-only restore changes and 10% lower peak RSS from streamed restore. A separate workload that included setup plus 100 four-view reports measured 19% lower wall time. These results are not additive and do not describe an ordinary one-shot command or Linux; the performance evidence preserves each workload and interval.

Install

uvx fdu@latest --help               # run the latest release without installing it
uv tool install fdu                 # the command line from a prebuilt wheel; uv tool upgrade fdu
cargo install --locked fdu          # the command line, built with Rust 1.85 or newer
pip install fdu                     # the Python package: import fdu
cargo add fdu                       # the Rust library, re-exported with the command line
cargo add fdu-core --features watch # the engine alone, with the watch layer

fdu --install-skill writes the agent skill to .agents/skills/fdu/SKILL.md and .claude/skills/fdu/SKILL.md under the project root, or to DIR/skills/fdu/SKILL.md with --agent-base DIR for one agent’s user scope; the skill prefers an fdu on PATH and otherwise runs uvx fdu@latest.

--locked builds against the lockfile published with the crate, which keeps the reviewed dependency set; see SUPPLY-CHAIN-SECURITY.md.

Platforms

Platform Command line and library Python wheel
macOS 11+, x86-64 and arm64 Supported; the scanner uses getattrlistbulk abi3 wheel
Linux glibc 2.17+, x86-64 and arm64 Supported abi3 wheel (manylinux2014); arm64 is cross-built and inspected, not run, before release
Windows x86-64 Supported, with no performance claims abi3 wheel
  • The Python package supports CPython 3.12 and newer through one stable-ABI (abi3) extension, tested on 3.12 and 3.14. Free-threaded CPython cannot load an abi3 extension, so it is not supported; use a GIL-enabled interpreter, as in uv tool install --python 3.14 fdu.
  • Other systems, including musl Linux and Windows arm64, build the source distribution with Rust 1.85 or newer.
  • Performance evidence comes mainly from an M1 Pro MacBook with a local APFS SSD; Linux measurements are from virtualized hosts, and the evidence report says what each figure is evidence about.

Compatibility

fdu is 0.x. A minor release (0.1 to 0.2) may change the Rust or Python API or the command line incompatibly, and its CHANGELOG names each change; a patch release never does. A machine consumer pins the schema string a report carries, which changes whenever a field does.

This release emits fdu.report/7 for every report, including the list rows above, fdu.stream/2 for a --watch stream, and fdu.cache/2 for cache status, which is its own document rather than a report and carries the identity of every tier each cached file holds. Treat an unrecognized value as a version you cannot parse rather than guessing at the fields.

Known Limitations

  • Memory. fdu builds an exact index that later questions reuse, and every report but an unfiltered --no-gitignore --view summary retains it, so peak memory grows with the entries retained. The 2026-09-16 tool comparison, on a generated 1,000,001-entry tree, measured fdu’s depth-one tree report, with its cache off, near 285 MiB of peak RSS, against 29 MiB for dumac, 21 MiB for dua, and 641 MiB for dust; each tool was invoked under a comparison contract that returned only a total. --no-gitignore --view summary retains no index: on that tree it took 4.876 s at 15.0 MiB, against 4.942 s at 285.7 MiB for the default --view summary, with identical totals. The...
Read more