Skip to content

v1.7.3 — a field named for symbols counts symbols

Choose a tag to compare

@ojassug ojassug released this 01 Sep 17:47
· 104 commits to main since this release
445b3ed

Read this if anything you own parses the trace.

DriftCoverage.symbolBearingItems changes meaning: it counted items an AST validator covered, and now counts items that actually yield symbols. The number moves on 254 of 580 corpus rows, always upward.

Numbered a patch because the optimized output does not move. outputSha is identical on all 580 rows, so the same command over the same input emits the same bytes on stdout. The change is confined to the diagnostic trace on stderr and in the MCP response.

If you consume only the optimized text, this release is inert for you. If you parse driftCoverage, treat it as breaking — the version number does not say so, and that is the one thing the digit understates.

What was wrong

The field was new Set(after.items.filter((i) => !unchecked.has(i.id))) — which is trace.astCoverage.checked arriving a second way, under a name asserting a fact about symbols that nothing had checked. tools/corpus-harness/measure.js recorded both, so every corpus run carried the same column twice under two names, and any comparison of them agreed by construction.

The two never had to agree: extractSymbols is regexes over content with no language gate, while selectValidator has four branches. Both directions are real:

  • Symbols without a validator. Go between §59 and §60 reported symbolsBefore >= 3 beside symbolBearingItems: 0 on all 80 frozen Go files. §60 gave Go a validator, which removed Go from that population without fixing the field.
  • A validator without symbols. Six of this repo's own barrels — src/index.ts, src/config/index.ts, src/core/model/index.ts, src/core/ledger/index.ts, src/bench/index.ts, src/bench/fixtures/index.ts — are validator-covered and yield no symbols. Those are precisely the files §28 and §33 exist to protect.

Why it survived: on a mixed bundle it was wrong in both directions at once, and the errors partly cancelled to a plausible-looking 2 where the truthful count was 3. Not obviously broken — quietly ratified, including by two tests that pinned the misnomer rather than the behaviour.

Measured

Corpus frozen at 1e0f71f: 290 files, 580 rows, ratio 0.3, both routes, 16 fields per row.

rows differing 254 of 580
fields differing symbolBearingItems only
direction 254 up, 0 down
rows with symbolsBefore > 0 beside symbolBearingItems: 0 254 → 0
outputSha identical on all 580

The differing buckets name the cause — tcl, c, shell, perl, rust, css, every language with no validator — plus 63 rows of typescript/stdin, this project's own language on its own second route.

extractItemSymbols splits the per-item body out so the count and symbolsBefore come from one pass; computed as a separate filter it cost a measured 19 ms of 196 ms on an 18-item bundle.

Also

Corrects a stale claim in CLAUDE.md: v1.7.0 was not the first npm release since 1.6.0 — it was never published either. v1.7.1 was. A tag in this repository does not imply a registry version; check npm view tokendamper version.

Verification

npm run typecheck, npm run lint, npm run build clean; 97 files / 867 tests passing. Built artifact reports 1.7.3 three ways.

Full changelog: https://github.com/Epichlo/TokenDamper/blob/v1.7.3/CHANGELOG.md