ci: verify the 0.9.20 upgrade path on every platform - #1036
Merged
Conversation
Everyone upgrading from the last published CLI crosses the session-cache v7 -> v9 re-layout (#1005/#1007) and the daily-cache v17 -> v19 re-derivation (#1015) on their first run. That path was covered by unit tests on one platform against caches the tests wrote themselves. `npm run verify:upgrade` (and the matching matrix job over {ubuntu, windows, macos} x node {22.13.0, 22}) instead installs the real codeburn@0.9.20 into an isolated global prefix, points it at a generated seven-provider corpus in a HOME whose path contains a space, and lets it write a genuine session-cache.v7.json + daily-cache.v17.json. This build then runs against that same cache dir, installed the same way, so dist/parse-worker.js has to resolve from an entry point outside the checkout. It asserts the v7 file is retired, the v9 envelope and shards publish, the daily history does not shrink, and per-provider calls/tokens/cost match the baseline exactly for claude, codex, gemini, kiro and cursor. grok is reported rather than asserted (its accounting changed in #1015) and dsh is required to be new. It also smoke-tests serve --stdio against the one-shot payloads, pins CODEBURN_PARSE_WORKERS to 0 and 3 and requires identical shards and payloads either way, and checks that a second run re-parses nothing.
Claude Code deletes its transcripts after ~30 days, so between one run and the next a day can go from fully sourced to PARTIALLY sourced. On such a day the daily cache re-derives a smaller slice from the surviving files and that slice REPLACES the baseline one rather than being unioned with it, so the aged-out portion is lost. A day that aged out completely is carried forward correctly, which is what makes this a hole in never-lose rather than a missing feature. The check ages the corpus the way retention does: two days keep a single anchoring transcript each, a third loses every one. It then compares the (date, provider) slices in daily-cache.v19.json against the ones the 0.9.20 baseline recorded in daily-cache.v17.json, requiring the partial days never to shrink and the fully sourceless day to come back exactly. This is a hard failure, not a note. It is expected to be red until the fix lands: on the generated corpus the two partial days currently lose 76.6% and 56.8% of their cost, while the fully sourceless control day returns to the cent. It runs last and on its own cache dir, so mutating the corpus cannot disturb the payload parity comparison. The shard-republication defect found earlier stays a note, now naming its pending follow-up issue.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A CI workflow (
upgrade-path.yml, matrix ubuntu/windows/macos × Node 22.13.0/22, on PRs touchingsrc/**+ dispatch) andnpm run verify:upgradethat runs the same script locally. It:codeburn@0.9.20into an isolated prefix and runs it →session-cache.v7.json+daily-cache.v17.jsonbaseline and payloads.npm pack→ isolated prefix, invoked through the npm bin symlink sodist/parse-worker.jsresolution is exercised) against the same cache dir → asserts v7 retired,session-cache.v9/+ envelope,daily-cache.v19.jsonre-derived with v17 kept, history days not shrunk.menubar-jsonandmodels, exits on stdin close.CODEBURN_PARSE_WORKERS=0vs3→ identical payloads and shard bodies, pool confirmed engaged.>=baseline and the fully-sourceless day==baseline at (date, provider) granularity. Failed withLOST 76.6%before daily-cache: never lose a day's history when its transcripts have partially aged out #1033; passes now.Local run on macOS/Node 26 against current main: PASSED, ~17 s.
Not covered (stated)
DSH
.zstdvariant (Node 22.13 lacks zstd — legs wouldn't be comparable); the worker auto-gate thresholds (workers are forced); the.cmdshim on Windows gets a--versionsmoke only (Node won't spawn.cmdwithout a shell); a synthetic corpus, not a real user cache — the real-cache verification was done manually on the maintainer's machine.