test(cli): every row kind arrives in any order - #774
Merged
Conversation
A re-read appends, so one session's lines sit in different orders on two machines. The report is meant to be blind to that, and the existing check is four records and a single reversal. Four records cannot cover what has been added since. `by_flow` now keys an interval-derived row on a `FlowInterval` object, a tool-stated one on the skill's name, and the remainder on a symbol; `by_agent` keys two of its three rows on symbols. Mixed key kinds in one Map are exactly where insertion order leaks into output, since rows are ranked by size and ties broken on the row's own key — and nothing failed when either tie-break was broken. Verified against the real thing before writing it: 30,222 records of a live sink, shuffled within every day file, produced a byte-identical report to the unshuffled run. Three consecutive runs were byte-identical, and all eleven `--axis` artefacts stable. fast-check over 300 permutations of a fixture carrying every row kind once, plus a second test asserting the fixture exercises all of them — a permutation of records that produce one kind of row proves nothing about mixed keys. The `pickDeterministically` mutation survived the first fixture, which had no two records sharing a `billed_request_id`, so the function was never reached. Adding the pair is what made that guard real. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VWNxk63AGKkqE8HRqHLjGp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A re-read appends, so one session's lines sit in different orders on two machines and nothing a consumer does controls it. The report is meant to be blind to that, and
cost-report.unit.test.tsalready checks it — with four records and a single reversal.Four records cannot cover what has been added since. Two axes now mix key kinds inside one
Map, which is exactly where insertion order leaks into output:by_flowFlowIntervalobjectby_flowby_flowby_agentby_agentby_agentRows are ranked by size and ties broken on the row's own key. A tie-break that forgets part of the key, or a survivor picked as
group[0], answers differently for the same records in a different order — and nothing failed when either was broken.Verified against the real thing first
30,222 records of a live sink, shuffled within every day file with a seeded shuffle, produced a byte-identical report to the unshuffled run (
md5equal). Three consecutive unshuffled runs were byte-identical too, and all eleven--axisartefacts stable across runs. The property holds at scale; this is the guard that keeps it.The test
fast-check over 300 permutations of a fixture carrying every row kind once: both flow kinds and the remainder, all three agent attributions, a named prompt and one that named none, two tools, a pair with identical figures only a tie-break can order, and two records sharing one
billed_request_idsopickDeterministicallyis actually reached.A second test asserts the fixture exercises what the property is about — a permutation of records that produce one kind of row proves nothing about mixed keys.
""— 1""— 1pickDeterministicallyreturnscandidates[0]— 1The third survived the first version of this fixture, which had no two records sharing a
billed_request_id— the function was never reached, so the guard was decorative. Adding the pair is what made it real, and the coverage test is what stops that regressing quietly.Gates: 3456 tests / 309 files,
tsc,biome ci(2 pre-existing warnings, 0 errors), knip, jscpd, bundle within budget, 0 broken links.🤖 Generated with Claude Code
https://claude.ai/code/session_01VWNxk63AGKkqE8HRqHLjGp