fix(metadata-protocol): bump write epoch on remote metadata mutation receipt

[os-musk]

Fixes #13609

Ruling executed — A′ (2026-09-03, comment 5528370129)

What stands. The 02:25Z ruling A (comment 5519392217): the metadata.mutated receipt path bumps the local write epoch. Confirmed by measurement (comment 5525867467): the bump at the receipt choke point turns both ⛔ UNBOUNDED arms into bounded-at-0 ms.

What is amended.

1. Site. packages/metadata-protocol/src/protocol.ts, inside applyRemoteMetadataMutation, after the registry-convergence branch and before notifyMutationListenersLocal (the 集群对端的元数据写入不失效本节点的 listCache / registry —— 收到广播的节点最长 30s 继续服务旧定义 #5109 invalidate-before-notify rule). The "or the cluster bridge plugin" alternative is withdrawn: measured to be pure boot-time wiring with no receipt path and no route to the epoch, and the authz template keeps its bump in the state owner too.
2. Spelling. metadata-protocol must not import @objectstack/objectql, so the bump goes through a structural helper beside readWriteEpoch in meta-overlay-cache.ts (bumpWriteEpoch(engine, 'remote') or the seat's equivalent), never a direct import of the epoch type.
3. Pins — three changes, authorized. The two ⛔ arms invert (four assertion-level UNBOUNDED readings, two arms), and the SCOPED-kernel arm's bound moves from TTL_MS (30 000) to 0 with its rationale rewritten to say why (on a scoped kernel the overlay cache is the only local source, and the bump retires it at convergence instead of letting it lapse). ⛔ It is not loosened to a less-than-or-equal comparison; the pin keeps asserting the exact number.

Fence. protocol.ts stays fenced behind PR #14908 as the engine seat's policy. […] permits the seat to branch now and land after #14908 merges — no race, no lifting of the fence from this seat.

Execution. domain:engine lane. Clause-②: no (per the original ruling). Fixes #13609 on the PR that lands it.

#14908 merged (d5cbb44f3) before this branch was cut, so the fence condition is satisfied.

The site (file:line, this branch)

packages/metadata-protocol/src/protocol.ts, applyRemoteMetadataMutation (private method, lines 5187–5220 on this branch):

} else {
    await this.restoreArtifactRegistryView(type, evt.name, orgId);
}
// [#13609] Retire this replica's overlay-row cache at the moment of
// convergence — never a direct `@objectstack/objectql` import, see
// `bumpWriteEpoch`'s header in `meta-overlay-cache.ts`.
bumpWriteEpoch(this.engine, 'remote');
this.notifyMutationListenersLocal({ ...evt, type });

Placed exactly where A′ says: after the registry-convergence if/else, before notifyMutationListenersLocal (#5109 invalidate-before-notify — a listener that re-reads through getMetaItems must not observe the healed registry through a still-stale overlay cache).

The helper, and why it is not a direct import

packages/metadata-protocol/src/meta-overlay-cache.ts gains bumpWriteEpoch, declared beside the existing readWriteEpoch, both now built on one shared, un-exported resolveWriteEpochSeam that validates the full { current, bump, subscribe } structural surface (mirrors isWriteEpochLike from @objectstack/objectql, without importing it — this package must not depend on @objectstack/objectql):

function resolveWriteEpochSeam(engine: unknown): WriteEpochSeam | undefined { /* … */ }

export function readWriteEpoch(engine: unknown): number | undefined {
  return resolveWriteEpochSeam(engine)?.current;
}

export function bumpWriteEpoch(engine: unknown, reason: string): number | undefined {
  return resolveWriteEpochSeam(engine)?.bump(reason);
}

bumpWriteEpoch is package-internal — not re-exported from src/index.ts (matching the existing metaOverlayCacheEntryCount's own "package-internal" precedent in the same file) — with its one caller being applyRemoteMetadataMutation. Clause-② re-derived on the final diff: no new exported symbol and no new key on a published payload — bumpWriteEpoch never crosses index.ts, so the mechanical floor's "no" holds, matching the ruling's expectation but re-checked rather than assumed.

Mirrors authz-invalidation-bridge.ts:71's epoch.bump('remote') on the identical write-epoch substrate (#11968) — the metadata cluster channel now gets the same bump the authz cluster channel already had; this closes an asymmetry between two existing paths, it does not add a new mechanism.

The three authorized pin changes, before → after

All three live in packages/metadata-protocol/src/protocol.datasource-delete-prolongation.test.ts (762 → ~790 lines; PR #14431's re-verification file, landed as a98b61b3e):

Arm	Before (unfixed)	After (this PR)
⛔→✅ "…but the peer's very next READ re-hydrates the deleted row…"	serves(peer) → true; registry.listItems('datasource') → ['billing_db']	serves(peer) → false; registry.listItems('datasource') → []
⛔→✅ "Arm B measured, door READ during the residue window"	prolongation → null (UNBOUNDED, past 10 windows); serves(peer) → true	prolongation → 0; serves(peer) → false
⭐ "SCOPED kernel" (third pin, ruling A′ §3)	prolongation → TTL_MS (30 000 ms)	prolongation → 0 — bound moved, not loosened to a less-than-or-equal comparison; rationale rewritten in-line to say why (the overlay cache is the scoped kernel's only local source, and the bump retires it at convergence instead of letting it lapse)

The four controls the ruling says stay as they are, do: Arm B's registry-heal assertion, the "door NOT read" bounded-by-one-TTL-window arm, Arm A's no-bridge control (still UNBOUNDED), and the negative control (analytics_db still served, billing_db served iff !shape.attach) — all unchanged and all still passing.

Test file's own docblock is updated in the same places the behaviour changed (the "Arms" narrative, "why the re-hydration happens" → "why it no longer does", the four-seam checklist's item 3, "NOT FIXED HERE" → "FIXED HERE"), while the methodology/topology/proxy-declaration sections are left untouched as accurate history.

packages/metadata-protocol/src/meta-overlay-cache.test.ts gets three new unit cases for bumpWriteEpoch itself (accepts the full seam and returns the new value; declines the same way readWriteEpoch does on no/partial seam; retires a live cache entry the same way a real engine write does) — direct coverage of the helper, independent of the end-to-end pin above.

Ablation — declared before running, then run

Predicted, in writing, before mutating: removing the bumpWriteEpoch(this.engine, 'remote') call turns exactly the three pins above RED — 5 assertion-level failures across 3 test cases — while the other 6 tests in the file stay GREEN (they never reach the receipt path, or never depended on the epoch).

Mutation, confirmed on disk before running: packages/metadata-protocol/src/protocol.ts's bump line replaced with a comment via a Python string-replace that asserted exactly 1 occurrence of the exact call-site text before replacing (a 0-hit replace would itself exit 0, so the count was asserted, not assumed); git diff --stat showed the expected 1-line change and a grep -c for the removed anchor confirmed 0 occurrences before the test run.

Observed (pnpm --filter @objectstack/metadata-protocol exec vitest run --maxWorkers=2 src/protocol.datasource-delete-prolongation.test.ts against the mutated source, no rebuild — this file resolves ./protocol.js to src/protocol.ts directly):

Tests  3 failed | 6 passed (9)

The 3 failures are exactly the 3 pins above (expected true to be false, expected null to be +0, expected 30000 to be +0) — the predicted set, exactly.

Restore, pinned to HEAD (never a bare git checkout --, which restores from a possibly-polluted index): git checkout HEAD -- packages/metadata-protocol/src/protocol.ts, proved by state, not exit code — git diff HEAD empty, git status --porcelain empty, and git hash-object on the restored file equal to git rev-parse HEAD:packages/metadata-protocol/src/protocol.ts (both 50125f18004684b9714cf1abcf00fe02613cf5d9). Re-ran the same test file against the restored source: Tests 9 passed (9) again.

The mutation/restore script ran under a shell trap on EXIT, INT and TERM that ran the restore command, anchored to REPO_ROOT="$(git rev-parse --show-toplevel)" (absolute path), so a cap-kill mid-mutation would still have restored the tree.

Live multi-node — NOT MEASURED, declared

No live cluster driver or multi-process deployment is reachable from this container. Everything measured above ran on the in-process two-instance harness protocol.datasource-delete-prolongation.test.ts already declares as the accepted proxy (PR #14431's shape: two protocol instances over one shared sys_metadata store, each with its own registry/overlay-cache/write-epoch, joined by a real cross-instance transport that delivers every publish to every subscriber, publisher included).

What the proxy leaves open, unchanged by this round: whether the QA deployment that originally reported the prolongation was running the shipped in-process memory cluster driver (no cross-process delivery — nothing was listening) or hit a second defect in a genuinely distributed driver (PR #13883's residual question). Reproduction detail stays withheld under the #9387 disclosure ruling.

Gates

node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands (no path args) derived the change set from git each time, re-run after every meaningful diff change:

• Initial derivation (pre-merge, 4 files): 34 commands.
• Post-merge derivation (5 files, origin/main merged in — main advances continuously, git fetch + re-derive run repeatedly rather than once): 42 commands — all 42 run; 38 pass, 1 real finding fixed (check-system-context-census.mjs: our import addition shifted a line-anchor by one line; repaired with the gate's own --fix, never by hand), 3 legitimate PREREQUISITE NOT MET/NOT MEASURED (all 3 are whole-tree gates needing artifacts this local run does not build: check-test-completeness.mjs wants a saved turbo run test log; check:pm-half-states's sibling check-half-states.mjs needs GitHub census read access; check:dual-build-cjs-loads needs a full-repo pnpm build across 49+ packages).
• Our own re-anchor fix touched content/docs/permissions/system-context.mdx, which pulled in the full docs-family gate set on the next derivation: 66 commands total, 24 newly implicated, all run — 22 pass immediately, 2 needed a rebuild first: @objectstack/spec's check:docs (spec's src/ had moved under the origin/main merge, per AGENTS.md §9/§10 — its json-schema/dist were stale relative to the merged source, not relative to anything we wrote — pnpm --filter @objectstack/spec build fixed it) and check:skill-examples (its client-SDK surface additionally needed @objectstack/client-react built, which pnpm's dependency selector resolved to 36 of 80 workspace packages via that package's devDependencies — unrelated collateral, not a sign of anything wrong on our diff). Both green after their respective rebuilds; re-ran with no path filter afterward and confirmed the derived command list was unchanged (no new families appeared).

Full verdict-by-verdict list (all 66 derived commands, run against the final tree) is in the dev report JSON. Final gate union run at 28cc78412 / ca9728a8a (the second commit is the census --fix alone); pnpm --filter @objectstack/metadata-protocol test and typecheck re-run once more at ca9728a8a (final HEAD): 160 test files / 2 skipped, 2366 tests / 10 skipped, all pass; typecheck clean.

node scripts/pm/check-governed-merges.mjs --test against the final changed-path list → NOT governed (0 of 6 paths hit GOVERNED_SURFACES) — ordinary queue landing applies.

Changeset

.changeset/metadata-protocol-remote-mutation-epoch-bump.md — patch for @objectstack/metadata-protocol (the only package touched). Not a declared-breaking change (no removed/renamed authorable key, no export change), so no ADR-0087 marker is required — confirmed by check-adr-0087-registration.mjs's own verdict: "this PR adds no declared-breaking changeset."

Docs correction (follow-up round, 2026-09-03) — the non-blocking review finding

Independent review passed every substantive point on the fix above and named one non-blocking finding: bumpWriteEpoch now retires the meta-overlay-cache overlay-read cache directly, on receipt of metadata.mutated — a behavior two hand-written doc passages, written before this fix, did not describe. Both named authz.invalidated as the only channel that retires that cache on a peer, with OS_METADATA_OVERLAY_CACHE_TTL_MS as the sole floor otherwise. That is now the wrong direction for the residue this PR closes.

Fixed, narrowing the paragraph's structure (still correct: there are several ways the cache gets retired, TTL is still the floor when none of them fire) rather than rewriting it:

• content/docs/concepts/metadata-lifecycle.mdx — the "Peers re-read from the shared database has one bound worth naming" callout. Added the metadata.mutated receipt path (applyRemoteMetadataMutation → bumpWriteEpoch) alongside authz.invalidated as a retirement path, and narrowed "a deployment with no distributed cluster driver attached" to "no cluster bridge attached at all — the in-process memory driver."
• content/docs/deployment/environment-variables.mdx — the OS_METADATA_OVERLAY_CACHE_TTL_MS row, same correction. Also found the identical claim on the OS_LOCALIZATION_CACHE_TTL_MS row (leg C of Design: cross-request caching for the authenticated request path (tranche 2 of #10757) — write-invalidation-first, short-TTL fallback, configurable staleness window #11633): its cache is keyed on the exact same shared engine.writeEpoch seam (packages/core/src/security/resolve-authz-context.ts's readWriteEpoch(ql), confirmed against write-epoch.ts's own "ONE invalidation signal... shared" contract) — so it is equally retired by the new bump, and its row named the same now-outdated bound. Fixed both rows.

content/docs/kernel/cluster.mdx was flagged by this PR's own docs-drift-check bot (comment 5530470960, via the applyRemoteMetadataMutation anchor) — read in full and left unchanged: its Lane 2 (metadata.mutated) section describes registry convergence and re-read semantics only, and makes no claim anywhere about the overlay-cache / write-epoch staleness bound this diff moves. Flagged is not falsified.

Wider-tree grep for every claim this diff moves (OS_METADATA_OVERLAY_CACHE_TTL_MS, authz.invalidated, "overlay cache", "write epoch", metadata.mutated, bumpWriteEpoch, applyRemoteMetadataMutation) across content/docs/**, with a positive control, found no other hits beyond the three pages above (references/api/protocol.mdx's "overlay" mentions are about org-scope precedence and ETags, unrelated to cross-replica cache timing; kernel/services-checklist.mdx's getMetaItems mention is an API-surface checklist row, unrelated to caching semantics).

No code, test, or changeset touched by this follow-up — the existing patch changeset for @objectstack/metadata-protocol already covers the behavior change; a documentation correction adds nothing to it. Gates re-derived on the final file list (node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands, no path args): 67 commands (one more than the 66 above — check:overlay-whitelist-table, newly implicated by the metadata-lifecycle.mdx edit) — all 67 run at the final pushed head; 63 pass, 4 legitimate PREREQUISITE NOT MET (same shape as above, plus check:type-check-debt, all needing a full-repo build this container doesn't have). check-governed-merges.mjs --test on the full 8-path change set → still NOT governed.

---

🤖 Generated with Claude Code

https://claude.ai/code/session_0112hMx9hjJ9BgB28X97DS68

[github-actions]

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/metadata-protocol, touching 6 documentable anchor(s).

3 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

• content/docs/concepts/metadata-lifecycle.mdx (via ObjectStackProtocolImplementation (symbol, a top-level class), applyRemoteMetadataMutation (symbol, a method of class ObjectStackProtocolImplementation), bumpWriteEpoch (symbol, a top-level function))
• content/docs/deployment/environment-variables.mdx (via applyRemoteMetadataMutation (symbol, a method of class ObjectStackProtocolImplementation), bumpWriteEpoch (symbol, a top-level function))
• content/docs/kernel/cluster.mdx (via applyRemoteMetadataMutation (symbol, a method of class ObjectStackProtocolImplementation))

⛔ 2 release-owned page(s) also name something this change touched. These are read-only:

• content/docs/releases/v16.mdx (via ObjectStackProtocolImplementation (symbol, a top-level class))
• content/docs/releases/v17.mdx (via ObjectStackProtocolImplementation (symbol, a top-level class))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see

• 2 name(s) were too generic to anchor anything (single lowercase words)
• the SDK route bridge reached 47 of 219 client-bound route-ledger rows — the other 172 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 172: 14 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
• a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 9 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json d17f352b1bb3e78682559e6f09bc48e77fda4dd5 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 969e1a5864b92c4dfcf9617efb151edbc434573e — the merge of head 8d1f11e7f828feb27341b2575ec021ef27a70f47 into base d17f352b1bb3e78682559e6f09bc48e77fda4dd5, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 969e1a5864b92c4dfcf9617efb151edbc434573e && git checkout 969e1a5864b92c4dfcf9617efb151edbc434573e
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin d17f352b1bb3e78682559e6f09bc48e77fda4dd5 8d1f11e7f828feb27341b2575ec021ef27a70f47 && git checkout -B drift-repro d17f352b1bb3e78682559e6f09bc48e77fda4dd5 && git merge --no-ff 8d1f11e7f828feb27341b2575ec021ef27a70f47

node scripts/docs-audit/affected-docs.mjs --json d17f352b1bb3e78682559e6f09bc48e77fda4dd5

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs d17f352b1bb3e78682559e6f09bc48e77fda4dd5 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

[os-musk]

Independent review PASS, one finding closed — ready and armed

An independent reviewer re-derived every substantive claim rather than confirming it, returned PASS, and raised one non-blocking finding. That finding was fixed in-branch and is now closed. All 37 check runs on head 8d1f11e7f are success or skipped, including Lint & Repo Gates (20:42:19) and Build Docs (20:32:49).

What the review re-proved rather than accepted

Clause-② = no, established from the published surface, not from the ruling. bumpWriteEpoch is a new exported symbol (meta-overlay-cache.ts:266), so the mechanical floor makes it presumptively yes — and it is not reachable: package.json's exports map has exactly one entry ("." → dist/index), no subpath export exists, and src/index.ts (217 lines, the sole barrel, all explicit named re-exports, no export *) contains zero references to meta-overlay-cache. Repo-wide there is no subpath import of it outside the package's own test. Same non-export pattern as the file's existing metaOverlayCacheEntryCount precedent.

The refactor is behaviour-preserving, byte-for-byte. readWriteEpoch was changed to delegate to the new resolveWriteEpochSeam; the validation body moved verbatim, and every input shape was enumerated against origin/main — engine null/non-object, no writeEpoch, non-object writeEpoch, partial seam (missing or wrong-typed current/bump/subscribe, same three checks in the same order), full valid shape. All 20 pre-existing cases untouched and green.

A′ clause by clause, proved from source: the bump sits at protocol.ts:5215, after the registry-convergence if/else (:5197-5211) and before notifyMutationListenersLocal (:5216) — the #5109 invalidate-before-notify order. No @objectstack/objectql import in either file.

The three authorized pin changes, and no others. Four assertion-level readings across the two ⛔ arms inverted; the SCOPED-kernel arm moved TTL_MS (30 000) → 0. ⛔ Nothing was loosened to <= — every one is still a literal-number .toBe(...). git diff on the test file shows no other expect(...) changed: the bridge-heal arm, the bounded-by-one-TTL arm, Arm A's no-bridge control (still UNBOUNDED) and the negative control are all untouched.

Ablation reproduced end to end, with the restore proved by state — git hash-object == git rev-parse HEAD:<path> == 50125f18… before and after, empty git diff HEAD, empty git status --porcelain. The three new unit tests are non-vacuous: the decisive one establishes a cache hit (call count unchanged across two reads), bumps, then asserts the count doubles — a no-op bumpWriteEpoch leaves it unchanged and the test fails.

Gates: 66 commands derived off the merge base with no path arguments; the reviewer went further than the author and completed a full workspace build (71/71 tasks) so that check:dual-build-cjs-loads and check:type-check-debt — left NOT MEASURED in the author's run — returned real green verdicts. Net 64/66 measured green, 2 legitimately NOT MEASURED for environment reasons (a saved turbo-test log; repo-scoped GitHub reads this container's proxy refuses). check-governed-merges --test: NOT governed.

The finding, and the fix that went past it

The reviewer found content/docs/concepts/metadata-lifecycle.mdx still saying a peer's write does not retire the overlay-read cache, with authz.invalidated or the TTL as the only paths — which this PR falsifies directly. Advisory-only in CI, but a landed fix leaving published docs asserting the opposite of the shipped behaviour is not something to defer.

The fix round went one page further than the finding, correctly. A wider-tree grep turned up a third, unflagged instance of the same stale claim: OS_LOCALIZATION_CACHE_TTL_MS's row in content/docs/deployment/environment-variables.mdx. That cache reads the same engine.writeEpoch seam, so it is equally retired by the new bump. Verified independently by this seat rather than taken from the report: packages/core/src/security/resolve-authz-context.ts:1139 reads (ql as { writeEpoch?: unknown }).writeEpoch, and LOCALIZATION_CACHE_TTL_ENV = 'OS_LOCALIZATION_CACHE_TTL_MS' is declared at :1097 of that same file.

And content/docs/kernel/cluster.mdx — flagged by the docs-drift bot alongside the first page — was read in full and left unchanged, because it makes no claim about the overlay-cache/write-epoch bound this diff moves. Flagged is not falsified; the bot's anchor list is a hint, and the decision has to come from the code.

The corrections state what is now also true without implying anything false: the authz.invalidated channel and the TTL are not removed by this fix and remain the paths for everything the metadata cluster channel does not carry.

Standing constraints honoured: the live multi-node run stays declared NOT MEASURED; PR #13883's residual question (shipped in-process memory driver vs. a second defect in a real distributed driver) stays named, not resolved; reproduction detail stays withheld under the #9387 disclosure ruling; the pin file's docblock preserves both the QA observation and the source counter-evidence — annotated, never deleted.

Changeset patch, unchanged and correct. Ready, auto-merge armed (method: MERGE, enabled at 20:57:28Z).

---

Generated by Claude Code