feat: make the COT / price seam explicit (ADR-0007 step 1)#55
Merged
Conversation
cotdata has two producers by design: the CFTC downloader (free, any OS) and the price producer (Norgate needs Windows). They share one manifest.json, and _touch_manifest is a read-modify-write, so two producers running against one store eventually lose an entry. That hazard just became reachable: a second producer host now exists. Two changes, no flag day. MANIFEST PER HALF, DUAL-WRITTEN Each domain declares its owning half (store._DOMAIN_HALF, half_for). Writes land in manifests/<half>.json AND the legacy top-level manifest.json. Readers merge: legacy first, per-half overlaid on top, so fresher half data wins and a half whose producer has not run yet still resolves from legacy. That ordering is what makes this safe today rather than after everyone migrates. A concurrent producer clobbering the shared aggregate no longer loses anything for a reader on this version, because the per-half file is authoritative. The legacy file exists only for consumers pinned to an older cotdata and is dropped in a follow-up once they have moved. half_for() raises on an undeclared domain, so adding one forces a decision about which side of the seam it belongs on rather than defaulting silently. HALF-SCOPED CLI ENTRY POINTS cotdata-cot and cotdata-prices run one half each and refuse the other's flags. cotdata-update still runs both for a single-machine deployment. --check and --reconcile are read-only and work from either. This is the operational half of the same hazard: a Windows price box that could also run --cot-all would become a second COT producer racing the first. Now it cannot, structurally. Verified against the real store (legacy-only, 241 manifest entries): reads correctly, writes nothing on read. Suite 127 passed (was 110), 17 new tests including a simulated concurrent clobber of the shared aggregate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
mspinola
added a commit
that referenced
this pull request
Jul 26, 2026
The scheduler wrappers called cotdata-update, which accepts every flag. Now that cotdata-cot and cotdata-prices exist (#55), the wrappers use the one matching their job, so a host is scoped by construction rather than by remembering. The failure this closes: a price box that can also run --cot-all is one edited line away from becoming a second COT producer racing the first, which is the manifest read-modify-write hazard the split exists to contain. cotdata-prices refuses --cot-all outright. Both platforms, since the two scheduling docs deliberately mirror each other: run-prices.cmd and run-prices.sh call cotdata-prices, run-cot.cmd and run-cot.sh call cotdata-cot. Each wrapper says why in a comment. Prose updated where it named the wrapper's binary specifically: the "two wrapper scripts" intro, the full-path-not-PATH note, the "Task Scheduler can't find" heading, the NDU interactive-session example, the logging variant, and the cron bare-environment note. References to cotdata-update that are genuinely about the general CLI (--check, --help, manual one-off runs, the setup guide's install verification) are left alone. Angle-bracket-free .cmd invariant preserved (#51). Suite 127 passed. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 1 of ADR-0007: make the COT / price seam explicit inside cotdata, before anything moves out.
Why now
cotdata has two producers by design: the CFTC downloader (free, any OS) and the price producer (Norgate needs Windows). They share one
manifest.json, and_touch_manifestis a read-modify-write, so two producers running against one store eventually lose an entry.That hazard was theoretical until a second producer host appeared. It isn't any more.
1. Manifest per half, dual-written
Each domain declares its owning half (
_DOMAIN_HALF,half_for). Writes land inmanifests/<half>.jsonand the legacy top-levelmanifest.json.Readers merge legacy first, then overlay per-half, so fresher half data wins and a half whose producer has not run yet still resolves from legacy.
That ordering is what makes this safe today rather than after everyone migrates. A concurrent producer clobbering the shared aggregate no longer loses anything for a reader on this version, because the per-half file is authoritative. There is a test that simulates exactly that clobber.
The legacy file exists only for consumers pinned to an older cotdata, and gets dropped in a follow-up once they have moved. No flag day, reversible.
half_for()raises on an undeclared domain, so adding one forces a decision about which side of the seam it belongs on rather than defaulting silently.2. Half-scoped CLI entry points
Each scoped entry point refuses the other half's flags:
This is the operational half of the same hazard. A Windows price box that could also run
--cot-allwould become a second COT producer racing the first. Now it cannot, structurally.--checkand--reconcileare read-only and work from either.Verification
cotdata-prices --cot-allis refused by the installed console script, not just in-process.Not in this PR
Dropping the legacy aggregate. That is the second half of step 1 and waits until both producers have run cleanly on this version.
🤖 Generated with Claude Code