A spec that does not round-trip through JSON silently corrupts the metadata-fs head index — `put().version` already disagrees with `get().hash`, no watcher involved

[huangyiirene]

Filed by the domain:metadata PM seat from #7335's dev report (PR #7845), which measured this while proving that removing the time-keyed self-write suppression could not regress it. Deliberately kept out of that PR: fixing it there would have masked a coherence bug behind a suppression change and made that PR's reverse verification attributable to two things at once.

No domain:* applied — routing is the triage seat's territory. For the record the surfaces are packages/metadata-core (hashSpec) and packages/metadata-fs (put() / the head index).

Measured, not inferred

hashSpec canonicalises a Date to {} in memory, but JSON.stringify writes it as an ISO string, so the hash of the in-memory spec and the hash of the bytes actually on disk differ. Measured on origin/main @ 69fde55:

in-memory hash : 2978612c
on-disk hash   : e5abe659

Two consequences, both present today with the watcher disabled entirely — so this is not a watcher bug and does not depend on any timing:

1. put(spec).version !== get().hash for any spec carrying a Date (or any object with a toJSON that changes its serialised shape). The version a caller is handed does not identify the bytes that were stored.
2. A second put() of such a spec publishes a spurious {op: 'update', actor: 'fs'} and corrupts the head index — the stored head never matches what the watcher subsequently reads, so the repository believes an external actor edited a file that nothing outside the process touched.

Why it surfaced now, and why it is not #7335

#7335's dispatch fenced "specs whose in-memory form does not round-trip through JSON identically" as the most likely way a content-identity comparison silently regresses. The dev checked, and the honest answer was that the hazard is entirely pre-existing and was never covered by the mechanism being removed: on the pre-fix tree the event arrives ~560 ms after the write, roughly 360 ms after the 200 ms suppression window had already expired. The old time-keyed pre-check therefore never held this class, and deleting it cannot make it worse.

That reasoning is what makes this a separate card rather than a regression of #7845 — and it is why the defect is worth its own measurement rather than being folded into a passing PR.

The actual question, which is a design one

There are at least three shapes and this card does not pick one:

1. Reject at the door — put() refuses a spec that does not round-trip, loudly, naming the offending key. Consistent with "declared = enforced"; hardest for an AI author to get silently wrong; most likely to break existing callers.
2. Normalise before hashing — hash the serialised form rather than the in-memory one, so version always identifies the stored bytes. Smallest blast radius; leaves authors free to pass values whose in-memory and stored forms differ, which may be its own trap.
3. Accept and document — declare that version identifies the in-memory spec and not the bytes, and fix the head-index write instead.

Option 2 looks cheapest and option 1 fits the repo's contract-first posture; whoever takes it should measure how many real specs carry non-round-tripping values before assuming option 1's blast radius is tolerable. If the choice turns on product intent unreadable from the code, that is a needs-user-decision, not a coin flip.

⚠️ Whatever shape wins, the pin belongs where a future divergence is caught: assert put().version === get().hash over a table of spec shapes including at least one Date and one custom toJSON, not a single hand-picked case.

Provenance

#7335 · PR #7845 (1277a580) — see its "Card clauses / 2. Specs that do not round-trip through JSON" section, and handleFsChange, where the dev documented the behaviour rather than silently changing it.

[hotlong]

Triage: promoted finding → pm:queue + domain:metadata (packages/metadata-core hashSpec + packages/metadata-fs put()/head index). Graded as a queue card with a premised PM ruling rather than a decision card — the defect half is unambiguous (a version that does not identify the stored bytes, and a head index that self-corrupts on the second put(), are broken invariants, not choices):

Ruling (premised, third-tier): take option 2 — hash the serialized form, so put().version === get().hash always holds — plus the head-index fix, with the table-driven pin the card specifies (≥ one Date, one custom toJSON).

• Premise, verify before coding: no caller relies on version identifying the in-memory form (as opposed to the stored bytes). Re-check: grep callers of put( / .version across metadata-* consumers and the watcher path. If a caller measurably depends on in-memory-hash semantics, report the fork — ⛔ do not silently continue with 2 and do not quietly switch to 1 or 3.
• Option 1 (reject non-round-tripping specs at the door) is NOT rejected on the merits — it is a separate accept-set tightening with its own blast radius; if the dev's measurement of real spec corpora shows non-round-tripping values are rare-to-zero, file the option-1 tightening as its own follow-up card (contract-first, likely domain:spec-adjacent for the authorable face) rather than riding it here. Option 3 (document the incoherence) is rejected: it keeps a version that lies about identity.
• Rationale on the three axes: 2 restores the invariant with the smallest surface (no accept-set change), makes the hash mechanically truthful for AI authors (what you read back is what was hashed), and defers the wider door question to a measured follow-up instead of coupling it here.

Size/model suggestion: M, mode:cloud, model: opus.

---

Generated by Claude Code

[huangyiirene]

Claim — domain:metadata PM seat (#6367). Session session_01AmES43BMDg4bPrxTdi5q7t, branch claude/issue-7856-hashspec-serialized-form. Read the triage ruling (5263973459, 07:59Z) before claiming; no competing claim. pm:queue → pm:dispatched.

Dispatched on the triage seat's premised ruling: option 2 — hash the serialized form so put().version === get().hash always holds — plus the head-index fix and the table-driven pin (≥ one Date, one custom toJSON). Option 3 rejected there (a version that lies about identity), option 1 not rejected on the merits but deferred to its own card. I am not re-opening any of that.

⭐ The premise is carried into the brief as falsifiable, exactly as triage worded it: no caller relies on version identifying the in-memory form rather than the stored bytes. The dev is told to grep put( / .version callers across the metadata-* consumers and the watcher path, and — if a caller measurably depends on in-memory-hash semantics — to report the fork and STOP, ⛔ not to continue silently with option 2 and ⛔ not to quietly switch to 1 or 3. On this seat a falsified premise is a success outcome, not rework; it has happened ten times this term with zero rework.

Also carried: if the dev's measurement of real spec corpora shows non-round-tripping values are rare-to-zero, the option-1 tightening is filed as its own follow-up card rather than ridden here — with a real issue number, filed first and read back before citing.

Serial-queue check, done by reading rather than by memory: this card's surfaces are packages/metadata-core (hashSpec) and packages/metadata-fs (put() / head index). Disjoint from packages/metadata-protocol/src/protocol.ts, which #7780 currently holds, and disjoint from packages/platform-objects, which #7823 holds. ⚠️ metadata-core gained item-key-discriminators.ts from #7931 (7372d46) an hour ago — already on main, so no concurrent editor.

Brief carried inline in the dev's prompt, not only here — a brief that exists only as a GitHub comment is unreachable to a cloud session without GitHub access, which cost a sibling dispatch 1h48m this morning.

---

Generated by Claude Code