Skip to content

feat: make the COT / price seam explicit (ADR-0007 step 1)#55

Merged
mspinola merged 1 commit into
mainfrom
feat/explicit-cot-price-seam
Jul 26, 2026
Merged

feat: make the COT / price seam explicit (ADR-0007 step 1)#55
mspinola merged 1 commit into
mainfrom
feat/explicit-cot-price-seam

Conversation

@mspinola

Copy link
Copy Markdown
Owner

Step 1 of ADR-0007: make the COT / price seam explicit inside cotdata, before anything moves out.

Why now

cotdata has two producers by design: the CFTC downloader (free, any OS) and the price producer (Norgate needs Windows). They share one manifest.json, and _touch_manifest is a read-modify-write, so two producers running against one store eventually lose an entry.

That hazard was theoretical until a second producer host appeared. It isn't any more.

1. Manifest per half, dual-written

Each domain declares its owning half (_DOMAIN_HALF, half_for). Writes land in manifests/<half>.json and the legacy top-level manifest.json.

Readers merge legacy first, then overlay per-half, so fresher half data wins and a half whose producer has not run yet still resolves from legacy.

That ordering is what makes this safe today rather than after everyone migrates. A concurrent producer clobbering the shared aggregate no longer loses anything for a reader on this version, because the per-half file is authoritative. There is a test that simulates exactly that clobber.

The legacy file exists only for consumers pinned to an older cotdata, and gets dropped in a follow-up once they have moved. No flag day, reversible.

half_for() raises on an undeclared domain, so adding one forces a decision about which side of the seam it belongs on rather than defaulting silently.

2. Half-scoped CLI entry points

cotdata-cot     --cot-all                              # CFTC half, any OS
cotdata-prices  --prices --metadata --require-final    # price half, Windows for Norgate
cotdata-update  ...                                    # both, single-machine deployment

Each scoped entry point refuses the other half's flags:

cotdata-prices: error: --cot-all belong(s) to the cot half. This entry point runs the
prices half only, so one host does one job. Use cotdata-cot for those, or cotdata-update
to run both from one machine.

This is the operational half of the same hazard. A Windows price box that could also run --cot-all would become a second COT producer racing the first. Now it cannot, structurally.

--check and --reconcile are read-only and work from either.

Verification

  • Real store (legacy-only, 241 manifest entries across 5 domains): reads correctly, writes nothing on read.
  • Suite: 127 passed, was 110. Seventeen new tests, including the simulated concurrent clobber, a legacy-only store, a mid-transition store where only one producer has run on the new code, and an assertion that every action flag is assigned to a half.
  • Verified cotdata-prices --cot-all is refused by the installed console script, not just in-process.

Not in this PR

Dropping the legacy aggregate. That is the second half of step 1 and waits until both producers have run cleanly on this version.

🤖 Generated with Claude Code

cotdata has two producers by design: the CFTC downloader (free, any OS) and the
price producer (Norgate needs Windows). They share one manifest.json, and
_touch_manifest is a read-modify-write, so two producers running against one
store eventually lose an entry. That hazard just became reachable: a second
producer host now exists.

Two changes, no flag day.

MANIFEST PER HALF, DUAL-WRITTEN

Each domain declares its owning half (store._DOMAIN_HALF, half_for). Writes land
in manifests/<half>.json AND the legacy top-level manifest.json. Readers merge:
legacy first, per-half overlaid on top, so fresher half data wins and a half
whose producer has not run yet still resolves from legacy.

That ordering is what makes this safe today rather than after everyone migrates.
A concurrent producer clobbering the shared aggregate no longer loses anything
for a reader on this version, because the per-half file is authoritative. The
legacy file exists only for consumers pinned to an older cotdata and is dropped
in a follow-up once they have moved.

half_for() raises on an undeclared domain, so adding one forces a decision about
which side of the seam it belongs on rather than defaulting silently.

HALF-SCOPED CLI ENTRY POINTS

cotdata-cot and cotdata-prices run one half each and refuse the other's flags.
cotdata-update still runs both for a single-machine deployment. --check and
--reconcile are read-only and work from either.

This is the operational half of the same hazard: a Windows price box that could
also run --cot-all would become a second COT producer racing the first. Now it
cannot, structurally.

Verified against the real store (legacy-only, 241 manifest entries): reads
correctly, writes nothing on read. Suite 127 passed (was 110), 17 new tests
including a simulated concurrent clobber of the shared aggregate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@mspinola
mspinola merged commit 9a5d712 into main Jul 26, 2026
5 checks passed
@mspinola
mspinola deleted the feat/explicit-cot-price-seam branch July 26, 2026 18:43
mspinola added a commit that referenced this pull request Jul 26, 2026
The scheduler wrappers called cotdata-update, which accepts every flag. Now that
cotdata-cot and cotdata-prices exist (#55), the wrappers use the one matching
their job, so a host is scoped by construction rather than by remembering.

The failure this closes: a price box that can also run --cot-all is one edited
line away from becoming a second COT producer racing the first, which is the
manifest read-modify-write hazard the split exists to contain. cotdata-prices
refuses --cot-all outright.

Both platforms, since the two scheduling docs deliberately mirror each other:
run-prices.cmd and run-prices.sh call cotdata-prices, run-cot.cmd and run-cot.sh
call cotdata-cot. Each wrapper says why in a comment.

Prose updated where it named the wrapper's binary specifically: the "two wrapper
scripts" intro, the full-path-not-PATH note, the "Task Scheduler can't find"
heading, the NDU interactive-session example, the logging variant, and the cron
bare-environment note. References to cotdata-update that are genuinely about the
general CLI (--check, --help, manual one-off runs, the setup guide's install
verification) are left alone.

Angle-bracket-free .cmd invariant preserved (#51). Suite 127 passed.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant