Skip to content

fdu 0.4.0

Choose a tag to compare

@jlevy jlevy released this 09 Oct 07:26
· 282 commits to main since this release
Immutable release. Only release title and notes can be modified.
c041ed1

fdu 0.4.0 changes what a report asks for and what it says. A content view now runs the analysis it shows: fdu . --view code,documents counts lines of code by language and words by document type in one scan, with no --analyze, where 0.3.0 refused a content view until you named its analyzer. A human report now prints only the result on standard output, and every explanation follows it on standard error as a short note: or tip: line, consolidated so that one note says what the totals include, one lists every display limit that hid something, and one runnable tip lifts them all. documents percentages now add up to 100%.

The release is breaking in a few places, listed in Upgrading from 0.3.0: --view=documents and --view=full beside --analyze=lines or --analyze=code, and four Rust API types. No option was removed, and the report, cache, and stream schemas are unchanged.

fdu 0.4.0 on the Linux kernel source: the usage tree, then lines of code by language and words by document type, first with fdu’s cache cleared and then from the cache

fdu . and then fdu . --view code,documents, cold and cached, on the Linux kernel source (1.7 GiB, 95,938 files), recorded in real time on an M1 Pro: one run of each command, not a benchmark. The full-resolution video, fdu-demo.mp4, is attached to this release.

What’s Changed

Features

  • Content views run the analysis they show. --view=code runs code analysis and --view=documents runs words analysis, so fdu . --view code,documents gives both rundowns from one scan. --analyze now adds analyzers beyond what the views imply: analysis shown in a metadata view (--analyze=code --view=languages), analysis run only to warm the content cache, or a wider set. Views with a metadata meaning (tree, languages, types, families, full) never imply analysis. In Python, fdu.report(root, Query(views=(View.CODE,))) runs code analysis the same way. An index from fdu.open or fdu.scan keeps exactly the analyzers it was opened with, and a view it cannot answer is refused with the analyzer named. In machine output, request.analyze lists every analyzer that ran, named or implied.

  • Notes and tips after the result. Human reports hold rows, column headings, and multi-view section headers; everything that explains them is a note: or tip: line on standard error after the result. That includes what used to sit inside the result: the percentage denominator (Percentage column: code lines is now note: percentages are shares of code lines), the row bound, share-floor omissions, Ranked by, and the code overview’s coverage. The notes are consolidated:

    • one note for what totals include;
    • one note listing every display limit that hid something, such as note: display limits: below 1% of root, depth 5;
    • one runnable tip that lifts them all, such as tip: show more: --min-share=0% --depth=all (in Python, min_share=0%, depth=all).

    Beside other views, a note names the section it is about: note: percentages are shares of code lines (CODE), document words (DOCUMENTS). --quiet still suppresses notes and tips. Grouped rows are narrower too: a row’s first line ends at its file count, and each further measure, such as lines, words and pages, or generated files, is a line of its own indented under it, so the report fdu . --view code,documents prints on the Linux kernel source fits in 88 columns, where 0.3.0’s widest DOCUMENTS row took 154 (the perf: line on standard error is still longer). Those rows no longer show 0.3.0’s documentation count (4,063 documentation), which counts files in a doc, docs, or documentation directory or named like a README, CHANGELOG, or CONTRIBUTING file: in DOCUMENTS that is nearly every file, and it read as a measure of the documents. JSON, JSONL, and YAML keep it.

  • Refusals name their remedy. A message about analysis names the change that fixes it and a route that accepts it: note: code analysis not shown by summary with tip: show it: --view code, or note: full omits code, documents without analysis with tip: include them: --analyze all. A refused read of an index names what it was opened with, and an opened root says it runs no analyzer.

Fixes

  • documents percentages add up to 100%. Each row’s share divided its document words by the total row’s, but logical words are derived after pooling, so a pooled total of mixed formats differed from the sum of its rows: with a long-token format such as HTML beside ordinary prose, 0.3.0’s shares summed to 101.6% on the Linux kernel source, and with very short tokens they could sum well below 100%. Shares in every grouped view now divide by the sum of the rows’ values, which is what share.denominator and the total row’s share carry in machine output. Byte, code-line, and raw-word shares are unchanged, since their rows already summed to their totals.

Guidelines and Content

  • Agent skill. The skill that --install-skill writes leads with the tree, the code and documents rundown, and JSON output; tells an agent to read the note: and tip: lines after a result; and quotes the section-named percentage note and the tip for a partly shown report.

Documentation

  • README. The homepage is restructured: it opens with the recording above, leads with the tree, then the code and documents rundown, and its Speed section says why fdu is fast and labels its macOS comparison as exploratory.

Install

uvx --no-build fdu@latest --help    # run the latest release without installing it
uv tool install --no-build fdu      # the command line from a prebuilt wheel
cargo install --locked fdu          # the command line, built with Rust 1.85 or newer
pip install fdu                     # the Python package: import fdu
cargo add fdu                       # the Rust library, re-exported with the command line
cargo add fdu-core --features watch # the engine alone, with the watch layer

Re-run uvx --no-build fdu@latest --install-skill after upgrading so the agent skill describes this release.

Compatibility

fdu is 0.x: a minor release may change the command line or the Rust or Python API incompatibly, and the CHANGELOG names each change. Reports stay fdu.report/10, cache status fdu.cache/3, and the watch stream fdu.stream/2. No command-line option was removed.

Upgrading from 0.3.0

  • First run scans cold. 0.4.0 reuses no snapshot or content sidecar written by 0.3.0, since the engine fingerprint includes the release version: each cached tree’s first run walks it, and its first analyzed run reads every file again.
  • --view=documents runs words analysis. Beside --analyze=lines or --analyze=code it now runs words analysis as well, because documents implies it. --analyze=lines --view=documents --format=json reports request.analyze: ["lines", "words"] and share_metric: "document_words" where 0.3.0 reported ["lines"] and raw_words; a script that read raw-word shares by document type reads document_words instead. On a held index with lines analysis only, --view=documents is refused.
  • --view=full without words analysis omits DOCUMENTS. Under --analyze=lines or --analyze=code, the report lists DOCUMENTS in omitted_views and says note: full omits documents without words analysis with tip: include them: --analyze all. A script that read that section adds words to --analyze.
  • Human output. Lines that explained a result, such as (2 of 15), Percentage column: …, and Ranked by …, are now note: lines on standard error, and a grouped row puts each measure after its file count on a line of its own and no longer shows the documentation count. A script that parsed them from standard output reads the machine formats instead, where a row’s detection.flags.documentation still holds that count.
  • Rust: query::Request. It gains a private field recording which analyzers the caller named and which views implied the rest. Outside the engine crate, build one with Request::new, Request::read, or Request::build rather than a struct literal; its public fields basis, query, and now are unchanged.
  • Rust: query::RequestError. It is now #[non_exhaustive]; an exhaustive match adds a wildcard arm. Its analysis refusals change shape: ViewNeedsContent(ViewSpec) and NeedsAnalyzer { item, analyzer } are replaced by ViewNeedsAnalyzer { view, held, holder } and SortNeedsAnalyzer { metric, analyzer, held, holder }, where holder is the new BasisHolder; WatchContent becomes WatchContent { named, views }; and AnalyzerNamedAsView::suggested_view is a ViewSpec (its label() is the old string). Each of these variants is #[non_exhaustive], so match them with ... Code that renders a refusal with message or Display is unaffected.
  • Rust: ViewSpec::full_report. Under AnalysisSet::LINES_ONLY or CODE_ONLY it returns Documents among the omitted views rather than the shown ones.
  • Rust: query::AxisNames. It gains a public field, setting_separator, which joins several settings in one suggestion.
  • Python. fdu.report with View.CODE or View.DOCUMENTS runs that analysis instead of refusing. index.report on an index opened without the analyzer raises InvalidArgumentError naming it; open with AnalysisOptions(analyze="code") or "words" to serve those views.

Known Limitations

  • DOCUMENTS totals are pooled. The total row’s document_words and pages are counted from statistics pooled across all files, so on a tree of mixed formats the rows can sum to more or less than the total row, while the percentages divide by the rows’ sum.
  • A watch’s remedy can point at a refusal. A watch checks a metric sort, and in Python a content view, before it refuses analysis: fdu . --watch --sort=code_lines says add --analyze code, and Index.watch() says its index was opened with analyze none, which sends the caller to an analyzer that a watch then refuses.
  • Performance. 0.4.0 changes how a report is requested and assembled, not the walk. No timing of 0.4.0 against 0.3.0 has been recorded; 0.3.0’s figures are in its release notes.
  • The 0.3.0 limitations on performance evidence, memory, and code analysis on a first run, and the 0.2.0 limitations on links, cache retention, watch backends, opened roots, and JSON integers, still apply.

Security

Report vulnerabilities through GitHub’s private vulnerability reporting, as SECURITY.md describes.

Full changelog: CHANGELOG.md

Full commit history: changes from v0.3.0 to v0.4.0