Skip to content

Persist unreadable/unwritable counts in the runs table so rebuilt exports can recover them #20

Description

@NemboKid

unreadable and unwritable are counted in the sweeping process and never stored — the module doc of crates/sweeper/src/bin/export-run.rs says exactly this, and it is why every rebuilt export writes them as null with a rebuilt_at marker rather than asserting zero. The cost is already visible in public: all four published archives in published-runs.json carry "unreadable": null, "unwritable": null, and the web /data page renders "unknown (rebuilt export)" for every one of them. The counts existed at sweep end; they were simply thrown away.

Scope: write them to the runs table when the sweep finishes, so a rebuild can recover them. The honesty rule stays: runs that predate the columns remain null, because for them "we no longer know" is still the truth.

Where to look

  • crates/sweeper/src/main.rs — the unreadable/unwritable counters and the sweep-end call that hands them to the export (around the "N unreadable, M unwritable this session" log)
  • crates/sweeper/src/store.rs — where the run row is finished; the natural place to persist the two counts
  • crates/sweeper/src/export.rs — the manifest fields and their "absent means unknown, not zero" semantics
  • crates/sweeper/src/bin/export-run.rs — the rebuild path whose module doc documents the current loss; it should read the stored counts once they exist
  • migrations/runs is created in 0008_conformance_ladder.sql and extended by 0009/0014/0016; follow that pattern with nullable columns

Done when

  • A migration adds nullable runs.unreadable and runs.unwritable
  • The sweeper writes both counts when it finishes a run
  • export-run puts the stored values into a rebuilt manifest, writing null only for runs that predate the columns
  • The export-run module doc's "counted in the sweeping process and never stored" claim is updated to describe the new behavior

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions