add http addr flag#9
Merged
Merged
Conversation
flemzord
pushed a commit
that referenced
this pull request
Mar 27, 2023
gfyrag
pushed a commit
that referenced
this pull request
Jul 2, 2026
…back Two related tightenings surfaced by review: 1. AssertPresent's Preload check is now strict Gen0-only via CacheSnapshotter.HasGen0Entry. The old HasEntry(gen0 or gen1) was too tolerant: admission's CacheGuaranteed verdict specifically asserted a Gen0 hit at case 0, and Path B (CheckCache case 1 + gen0-hit → Touch, not AssertPresent) guarantees no rotation between admission and apply for AssertPresent-emitting cases. A gen1-only state at Preload therefore signals a real bug (torn CheckCache read, predicted_index drift, Path B failure) — retryable ErrStalePreload is safer than proceeding, because letting the order run on a gen1 fallback would panic strict Del if the same order both reads AND deletes the key. A gen0 tombstone still counts as present (Scope.GetX sees the Deleted flag and returns ErrNotFound cleanly — legitimate delete between admission and apply, no divergence). 2. Remove the gen0→gen1 fallback in AttributeCache.Get. The fallback silently absorbed invariant #6 violations: a read on a key the proposer never preloaded that happened to still sit in Gen1 (pre- rotation legacy) would succeed instead of surfacing as ErrNotFound. Callers that legitimately need to consult Gen1 (MirrorPreload's gen1-wins, HasLiveEntry, snapshotter persistence) already use explicit Gen1() access. The only production caller of Get was KeyStore.Get / GetEntry / Delete — these become strict Gen0-only. Concretely for KeyStore.Delete on a gen1-only key: previously it fetched via fallback, passed the existence check, then panicked at strict AttributeCache.Del. Now it returns ErrNotFound early — safer contract on the primitive. The underlying invariant #6 detection moves to the coverage gate (invariant #9) via Scope.GetX, which is where preload-declaration enforcement belongs. Tests updated: - TestAttributeCache_Rotate / TestCache_CheckRotationNeeded_* — assert strict Gen0-only Get semantics; use ac.Gen1().Get(...) for post- rotation entry access. - TestCache_AllAttributeCachesRotate — same shift. - TestKeyStoreDelete_PreloadContractViolation → renamed TestKeyStoreDelete_Gen1OnlyReturnsNotFound — pins the new clean- ErrNotFound contract on gen1-only Delete. - TestPreload_AssertPresent_AcceptsGen1Fallback → renamed / rewritten TestPreload_AssertPresent_RejectsGen1Only — matches strict Gen0 semantic. - New TestCacheSnapshotter_HasGen0Entry (replaces HasEntry test) — pins the four cases (gen0 live / gen0 tombstone / gen1 live / gen1 tombstone / absent). Doc: docs/technical/architecture/core/plan-intent-verification.md — the intent table now shows HasGen0Entry (strict) for AssertPresent, with the rationale for choosing Gen0-only over the fallback semantics.
gfyrag
pushed a commit
that referenced
this pull request
Jul 2, 2026
…sent/ExpectAbsent Replace per-intent Preload verification (AssertPresent/ExpectAbsent/Touch) with a single try-promote pass. Every non-Value plan entry is now emitted as coverage-only Declare and the FSM's MirrorTouch handles gen1→gen0 promotion uniformly at Preload — gen0 hit no-ops, gen1 hit promotes, neither silently no-ops. Correctness rests on two orthogonal guardrails, not per-intent checks: - Admission-side CacheUnreachable (PR #1458): 2+ predicted rotations reject the proposal with ErrCacheHorizonExceeded, bounding the propose→apply window to at most one rotation. - Coverage gate (invariant #9): the FSM only reads keys admission declared, so Declare + MirrorTouch is a complete substitute for the historical assertion machinery. AttributeCache.Get is gen0-strict. Every declared key that exists somewhere in cache is promoted to gen0 by MirrorTouch before the handler runs, so the historical gen1 fallback in Get is redundant; handler reads see the fresh value directly. Deletes on genuinely absent keys now surface as clean ErrNotFound at KeyStore.Delete rather than the strict-Del contract-violation panic. Removed: Needs.AddExpectAbsent, AttributeSet.ExpectAbsent, all three assertion intents from the resolver's emission path, plan.ErrStalePreload sentinel + ErrStalePreloadDescribable adapter, CacheSnapshotter's HasGen0Entry / HasLiveEntry, and machine.go's per-intent Preload switch. The proto oneof variants (Touch / AssertPresent / ExpectAbsent) remain for wire compat but the resolver never emits them. Docs: plan-intent-verification.md rewritten for the unified model.
4 tasks
gfyrag
pushed a commit
that referenced
this pull request
Jul 2, 2026
…as promote primitive Restores strict AttributeCache.Del (Gen0 must hold the key at delete time) by shifting the invariant from "admission emits per-intent assertions" to "Preload uniformly promotes gen1→gen0 for every declared key". The propose→apply race window is bounded by the admission-side CacheUnreachable guard (2+ predicted rotations rejected up front, PR #1458), and the FSM coverage gate (invariant #9) binds the read horizon to admission's declared preload set — those two guardrails make Declare + MirrorTouch a complete substitute for the historical AssertPresent / ExpectAbsent / Touch machinery. Semantic changes: - `AttributeCache.Get` is gen0-strict — no gen1 fallback. `MirrorTouch` promotes any gen1 entry into gen0 at Preload before handlers run. - `AttributeCache.Del` returns an error (kv.KV signature updated) and refuses to fabricate a tombstone from Gen1. Preload-contract violations bubble up loudly per invariant #7 instead of silently desyncing. - `writeCacheTombstone` writes only the current gen0 byte in 0xFF (the gen1 byte is left as harmless stale data, purged on the next rotation) — keeping cache mem/disk equal for the same applied index (invariant #1). - `protoSnapshotSlot.MirrorTouch` is a silent no-op when neither gen has the key. Under the coverage gate that state is legitimate: a Declare entry may cover a key that is only in Pebble (Value intent seeds those) or genuinely absent (handler ErrNotFound path). - Resolver: `CacheGuaranteed` and `CacheNeedsTouch` both emit `Declare` now; `CacheMiss + Pebble-load-hit` still emits `Value(v)`. The proto variants `Touch` / `AssertPresent` / `ExpectAbsent` stay in raft_cmd.proto for wire compatibility with pre-EN-1242 plans but the resolver never emits them and the FSM Preload switch treats every non-Value intent identically via `MirrorTouch`. Removed machinery: - `plan.ErrStalePreload` sentinel (moved to a leaf `planerr` package so admission and state can reference `ErrCacheHorizonExceeded` without a cycle — that's all `planerr` holds now). - `CacheSnapshotter.HasGen0Entry` / `HasLiveEntry`. - The `touch_noop` panic that fired when `MirrorTouch` couldn't find the key in either generation. Docs: `docs/technical/architecture/core/plan-intent-verification.md` rewritten for the unified model — problem statement, Path A / Path B race traces, the two guardrails, and the delete-site checklist. Layered on top of PR #1475 (Needs generic dispatch refactor).
gfyrag
pushed a commit
that referenced
this pull request
Jul 2, 2026
…as promote primitive Restores strict AttributeCache.Del (Gen0 must hold the key at delete time) by shifting the invariant from "admission emits per-intent assertions" to "Preload uniformly promotes gen1→gen0 for every declared key". The propose→apply race window is bounded by the admission-side CacheUnreachable guard (2+ predicted rotations rejected up front, PR #1458), and the FSM coverage gate (invariant #9) binds the read horizon to admission's declared preload set — those two guardrails make Declare + MirrorTouch a complete substitute for the historical AssertPresent / ExpectAbsent / Touch machinery. Semantic changes: - `AttributeCache.Get` is gen0-strict — no gen1 fallback. `MirrorTouch` promotes any gen1 entry into gen0 at Preload before handlers run. - `AttributeCache.Del` returns an error (kv.KV signature updated) and refuses to fabricate a tombstone from Gen1. Preload-contract violations bubble up loudly per invariant #7 instead of silently desyncing. - `writeCacheTombstone` writes only the current gen0 byte in 0xFF (the gen1 byte is left as harmless stale data, purged on the next rotation) — keeping cache mem/disk equal for the same applied index (invariant #1). - `protoSnapshotSlot.MirrorTouch` is a silent no-op when neither gen has the key. Under the coverage gate that state is legitimate: a Declare entry may cover a key that is only in Pebble (Value intent seeds those) or genuinely absent (handler ErrNotFound path). - Resolver: `CacheGuaranteed` and `CacheNeedsTouch` both emit `Declare` now; `CacheMiss + Pebble-load-hit` still emits `Value(v)`. The proto variants `Touch` / `AssertPresent` / `ExpectAbsent` stay in raft_cmd.proto for wire compatibility with pre-EN-1242 plans but the resolver never emits them and the FSM Preload switch treats every non-Value intent identically via `MirrorTouch`. Removed machinery: - `plan.ErrStalePreload` sentinel (moved to a leaf `planerr` package so admission and state can reference `ErrCacheHorizonExceeded` without a cycle — that's all `planerr` holds now). - `CacheSnapshotter.HasGen0Entry` / `HasLiveEntry`. - The `touch_noop` panic that fired when `MirrorTouch` couldn't find the key in either generation. Docs: `docs/technical/architecture/core/plan-intent-verification.md` rewritten for the unified model — problem statement, Path A / Path B race traces, the two guardrails, and the delete-site checklist. Layered on top of PR #1475 (Needs generic dispatch refactor).
gfyrag
pushed a commit
that referenced
this pull request
Jul 2, 2026
…as promote primitive Restores strict AttributeCache.Del (Gen0 must hold the key at delete time) by shifting the invariant from "admission emits per-intent assertions" to "Preload uniformly promotes gen1→gen0 for every declared key". The propose→apply race window is bounded by the admission-side CacheUnreachable guard (2+ predicted rotations rejected up front, PR #1458), and the FSM coverage gate (invariant #9) binds the read horizon to admission's declared preload set — those two guardrails make Declare + MirrorTouch a complete substitute for the historical AssertPresent / ExpectAbsent / Touch machinery. Semantic changes: - `AttributeCache.Get` is gen0-strict — no gen1 fallback. `MirrorTouch` promotes any gen1 entry into gen0 at Preload before handlers run. - `AttributeCache.Del` returns an error (kv.KV signature updated) and refuses to fabricate a tombstone from Gen1. Preload-contract violations bubble up loudly per invariant #7 instead of silently desyncing. - `writeCacheTombstone` writes only the current gen0 byte in 0xFF (the gen1 byte is left as harmless stale data, purged on the next rotation) — keeping cache mem/disk equal for the same applied index (invariant #1). - `protoSnapshotSlot.MirrorTouch` is a silent no-op when neither gen has the key. Under the coverage gate that state is legitimate: a Declare entry may cover a key that is only in Pebble (Value intent seeds those) or genuinely absent (handler ErrNotFound path). - Resolver: `CacheGuaranteed` and `CacheNeedsTouch` both emit `Declare` now; `CacheMiss + Pebble-load-hit` still emits `Value(v)`. The proto variants `Touch` / `AssertPresent` / `ExpectAbsent` stay in raft_cmd.proto for wire compatibility with pre-EN-1242 plans but the resolver never emits them and the FSM Preload switch treats every non-Value intent identically via `MirrorTouch`. Removed machinery: - `plan.ErrStalePreload` sentinel (moved to a leaf `planerr` package so admission and state can reference `ErrCacheHorizonExceeded` without a cycle — that's all `planerr` holds now). - `CacheSnapshotter.HasGen0Entry` / `HasLiveEntry`. - The `touch_noop` panic that fired when `MirrorTouch` couldn't find the key in either generation. Docs: `docs/technical/architecture/core/plan-intent-verification.md` rewritten for the unified model — problem statement, Path A / Path B race traces, the two guardrails, and the delete-site checklist. Layered on top of PR #1475 (Needs generic dispatch refactor).
gfyrag
pushed a commit
that referenced
this pull request
Jul 2, 2026
…as promote primitive
Fixes a strict AttributeCache.Del contract violation on Delete-like
orders when a concurrent Save + rotation runs between propose and apply:
T0 Admission: CheckCache(k) → CacheMiss → Declare (no seed)
T1 Concurrent Save(k, v) → Gen0[k] = v
T2 Cache rotation → Gen1[k] = v, Gen0[k] = ∅
T3 Delete(k) applies → strict Del sees Gen0∅ → PANIC
The fix relies on two orthogonal guardrails already in force on
release/v3.0:
1. Admission-side CacheUnreachable rejects any proposal predicting
2+ rotations between propose and apply (plan.ErrCacheHorizonExceeded,
gRPC Unavailable / HTTP 503). This bounds the propose→apply race
to at most one rotation, so any key admission observed anywhere in
cache is still reachable via Gen1 at Preload.
2. The coverage gate (invariant #9) binds every FSM cache read to
admission's declared preload set — a Declare entry can only be
read by an order that emitted it.
Given those bounds, Preload-time MirrorTouch is a complete substitute
for per-intent verification: for every Declare plan entry, promote any
Gen1 hit into Gen0 before the handler runs. Applied to T3 above, Gen0
gets seeded with v before strict Del runs and the panic is gone.
Semantic changes on the release/v3.0 baseline:
- AttributeCache.Get is gen0-strict — no gen1 fallback. MirrorTouch has
already promoted anything that exists anywhere in cache before the
handler reads.
- AttributeCache.Del returns an error (kv.KV signature updated) and
refuses to fabricate a tombstone from Gen1. Preload-contract
violations bubble up loudly per invariant #7 instead of silently
desyncing.
- writeCacheTombstone writes only the current gen0 byte in 0xFF (the
gen1 byte is left as harmless stale data, purged on the next
rotation) — keeping cache mem/disk equal for the same applied index
(invariant #1).
- protoSnapshotSlot.MirrorTouch is a silent no-op when neither gen has
the key (previous behavior: touch_noop panic). Under the coverage
gate that state is legitimate.
- Resolver: CacheGuaranteed and CacheNeedsTouch both emit Declare now;
CacheMiss + Pebble-load-hit still emits Value(v). The proto's Touch
variant is retained for wire compat with pre-refactor plans still in
flight during a rolling upgrade (FSM Preload treats it identically to
Declare via MirrorTouch).
Docs: docs/technical/architecture/core/plan-intent-verification.md
walks the Path A race, the two guardrails, and the strict-Del site
checklist.
Layered on top of PR #1475 (Needs generic dispatch refactor).
gfyrag
pushed a commit
that referenced
this pull request
Jul 2, 2026
…as promote primitive
Fixes a strict AttributeCache.Del contract violation on Delete-like
orders when a concurrent Save + rotation runs between propose and apply:
T0 Admission: CheckCache(k) → CacheMiss → Declare (no seed)
T1 Concurrent Save(k, v) → Gen0[k] = v
T2 Cache rotation → Gen1[k] = v, Gen0[k] = ∅
T3 Delete(k) applies → strict Del sees Gen0∅ → PANIC
The fix relies on two orthogonal guardrails already in force on
release/v3.0:
1. Admission-side CacheUnreachable rejects any proposal predicting
2+ rotations between propose and apply (plan.ErrCacheHorizonExceeded,
gRPC Unavailable / HTTP 503). This bounds the propose→apply race
to at most one rotation, so any key admission observed anywhere in
cache is still reachable via Gen1 at Preload.
2. The coverage gate (invariant #9) binds every FSM cache read to
admission's declared preload set — a Declare entry can only be
read by an order that emitted it.
Given those bounds, Preload-time MirrorTouch is a complete substitute
for per-intent verification: for every Declare plan entry, promote any
Gen1 hit into Gen0 before the handler runs. Applied to T3 above, Gen0
gets seeded with v before strict Del runs and the panic is gone.
Semantic changes on the release/v3.0 baseline:
- AttributeCache.Get is gen0-strict — no gen1 fallback. MirrorTouch has
already promoted anything that exists anywhere in cache before the
handler reads.
- AttributeCache.Del returns an error (kv.KV signature updated) and
refuses to fabricate a tombstone from Gen1. Preload-contract
violations bubble up loudly per invariant #7 instead of silently
desyncing.
- writeCacheTombstone writes only the current gen0 byte in 0xFF (the
gen1 byte is left as harmless stale data, purged on the next
rotation) — keeping cache mem/disk equal for the same applied index
(invariant #1).
- protoSnapshotSlot.MirrorTouch is a silent no-op when neither gen has
the key (previous behavior: touch_noop panic). Under the coverage
gate that state is legitimate.
- Resolver: CacheGuaranteed and CacheNeedsTouch both emit Declare now;
CacheMiss + Pebble-load-hit still emits Value(v). The proto's Touch
variant is retained for wire compat with pre-refactor plans still in
flight during a rolling upgrade (FSM Preload treats it identically to
Declare via MirrorTouch).
Docs: docs/technical/architecture/core/plan-intent-verification.md
walks the Path A race, the two guardrails, and the strict-Del site
checklist.
Layered on top of PR #1475 (Needs generic dispatch refactor).
gfyrag
pushed a commit
that referenced
this pull request
Jul 2, 2026
…as promote primitive
Fixes a strict AttributeCache.Del contract violation on Delete-like
orders when a concurrent Save + rotation runs between propose and apply:
T0 Admission: CheckCache(k) → CacheMiss → Declare (no seed)
T1 Concurrent Save(k, v) → Gen0[k] = v
T2 Cache rotation → Gen1[k] = v, Gen0[k] = ∅
T3 Delete(k) applies → strict Del sees Gen0∅ → PANIC
The fix relies on two orthogonal guardrails already in force on
release/v3.0:
1. Admission-side CacheUnreachable rejects any proposal predicting
2+ rotations between propose and apply (plan.ErrCacheHorizonExceeded,
gRPC Unavailable / HTTP 503). This bounds the propose→apply race
to at most one rotation, so any key admission observed anywhere in
cache is still reachable via Gen1 at Preload.
2. The coverage gate (invariant #9) binds every FSM cache read to
admission's declared preload set — a Declare entry can only be
read by an order that emitted it.
Given those bounds, Preload-time MirrorTouch is a complete substitute
for per-intent verification: for every Declare plan entry, promote any
Gen1 hit into Gen0 before the handler runs. Applied to T3 above, Gen0
gets seeded with v before strict Del runs and the panic is gone.
Semantic changes on the release/v3.0 baseline:
- AttributeCache.Get is gen0-strict — no gen1 fallback. MirrorTouch has
already promoted anything that exists anywhere in cache before the
handler reads.
- AttributeCache.Del returns an error (kv.KV signature updated) and
refuses to fabricate a tombstone from Gen1. Preload-contract
violations bubble up loudly per invariant #7 instead of silently
desyncing.
- writeCacheTombstone writes only the current gen0 byte in 0xFF (the
gen1 byte is left as harmless stale data, purged on the next
rotation) — keeping cache mem/disk equal for the same applied index
(invariant #1).
- protoSnapshotSlot.MirrorTouch is a silent no-op when neither gen has
the key (previous behavior: touch_noop panic). Under the coverage
gate that state is legitimate.
- Resolver: CacheGuaranteed and CacheNeedsTouch both emit Declare now;
CacheMiss + Pebble-load-hit still emits Value(v). The proto's Touch
variant is retained for wire compat with pre-refactor plans still in
flight during a rolling upgrade (FSM Preload treats it identically to
Declare via MirrorTouch).
Docs: docs/technical/architecture/core/plan-intent-verification.md
walks the Path A race, the two guardrails, and the strict-Del site
checklist.
Layered on top of PR #1475 (Needs generic dispatch refactor).
gfyrag
pushed a commit
that referenced
this pull request
Jul 2, 2026
…as promote primitive
Fixes a strict AttributeCache.Del contract violation on Delete-like
orders when a concurrent Save + rotation runs between propose and apply:
T0 Admission: CheckCache(k) → CacheMiss → Declare (no seed)
T1 Concurrent Save(k, v) → Gen0[k] = v
T2 Cache rotation → Gen1[k] = v, Gen0[k] = ∅
T3 Delete(k) applies → strict Del sees Gen0∅ → PANIC
The fix relies on two orthogonal guardrails already in force on
release/v3.0:
1. Admission-side CacheUnreachable rejects any proposal predicting
2+ rotations between propose and apply (plan.ErrCacheHorizonExceeded,
gRPC Unavailable / HTTP 503). This bounds the propose→apply race
to at most one rotation, so any key admission observed anywhere in
cache is still reachable via Gen1 at Preload.
2. The coverage gate (invariant #9) binds every FSM cache read to
admission's declared preload set — a Declare entry can only be
read by an order that emitted it.
Given those bounds, Preload-time MirrorTouch is a complete substitute
for per-intent verification: for every Declare plan entry, promote any
Gen1 hit into Gen0 before the handler runs. Applied to T3 above, Gen0
gets seeded with v before strict Del runs and the panic is gone.
Semantic changes on the release/v3.0 baseline:
- AttributeCache.Get is gen0-strict — no gen1 fallback. MirrorTouch has
already promoted anything that exists anywhere in cache before the
handler reads.
- AttributeCache.Del returns an error (kv.KV signature updated) and
refuses to fabricate a tombstone from Gen1. Preload-contract
violations bubble up loudly per invariant #7 instead of silently
desyncing.
- writeCacheTombstone writes only the current gen0 byte in 0xFF (the
gen1 byte is left as harmless stale data, purged on the next
rotation) — keeping cache mem/disk equal for the same applied index
(invariant #1).
- protoSnapshotSlot.MirrorTouch is a silent no-op when neither gen has
the key (previous behavior: touch_noop panic). Under the coverage
gate that state is legitimate.
- Resolver: CacheGuaranteed and CacheNeedsTouch both emit Declare now;
CacheMiss + Pebble-load-hit still emits Value(v). The proto's Touch
variant is retained for wire compat with pre-refactor plans still in
flight during a rolling upgrade (FSM Preload treats it identically to
Declare via MirrorTouch).
Docs: docs/technical/architecture/core/plan-intent-verification.md
walks the Path A race, the two guardrails, and the strict-Del site
checklist.
Layered on top of PR #1475 (Needs generic dispatch refactor).
gfyrag
pushed a commit
that referenced
this pull request
Jul 2, 2026
…as promote primitive
Fixes a strict AttributeCache.Del contract violation on Delete-like
orders when a concurrent Save + rotation runs between propose and apply:
T0 Admission: CheckCache(k) → CacheMiss → Declare (no seed)
T1 Concurrent Save(k, v) → Gen0[k] = v
T2 Cache rotation → Gen1[k] = v, Gen0[k] = ∅
T3 Delete(k) applies → strict Del sees Gen0∅ → PANIC
The fix relies on two orthogonal guardrails already in force on
release/v3.0:
1. Admission-side CacheUnreachable rejects any proposal predicting
2+ rotations between propose and apply (plan.ErrCacheHorizonExceeded,
gRPC Unavailable / HTTP 503). This bounds the propose→apply race
to at most one rotation, so any key admission observed anywhere in
cache is still reachable via Gen1 at Preload.
2. The coverage gate (invariant #9) binds every FSM cache read to
admission's declared preload set — a Declare entry can only be
read by an order that emitted it.
Given those bounds, Preload-time MirrorTouch is a complete substitute
for per-intent verification: for every Declare plan entry, promote any
Gen1 hit into Gen0 before the handler runs. Applied to T3 above, Gen0
gets seeded with v before strict Del runs and the panic is gone.
Semantic changes on the release/v3.0 baseline:
- AttributeCache.Get is gen0-strict — no gen1 fallback. MirrorTouch has
already promoted anything that exists anywhere in cache before the
handler reads.
- AttributeCache.Del returns an error (kv.KV signature updated) and
refuses to fabricate a tombstone from Gen1. Preload-contract
violations bubble up loudly per invariant #7 instead of silently
desyncing.
- writeCacheTombstone writes only the current gen0 byte in 0xFF (the
gen1 byte is left as harmless stale data, purged on the next
rotation) — keeping cache mem/disk equal for the same applied index
(invariant #1).
- protoSnapshotSlot.MirrorTouch is a silent no-op when neither gen has
the key (previous behavior: touch_noop panic). Under the coverage
gate that state is legitimate.
- Resolver: CacheGuaranteed and CacheNeedsTouch both emit Declare now;
CacheMiss + Pebble-load-hit still emits Value(v). The proto's Touch
variant is retained for wire compat with pre-refactor plans still in
flight during a rolling upgrade (FSM Preload treats it identically to
Declare via MirrorTouch).
Docs: docs/technical/architecture/core/plan-intent-verification.md
walks the Path A race, the two guardrails, and the strict-Del site
checklist.
Layered on top of PR #1475 (Needs generic dispatch refactor).
gfyrag
pushed a commit
that referenced
this pull request
Jul 2, 2026
…as promote primitive
Fixes a strict AttributeCache.Del contract violation on Delete-like
orders when a concurrent Save + rotation runs between propose and apply:
T0 Admission: CheckCache(k) → CacheMiss → Declare (no seed)
T1 Concurrent Save(k, v) → Gen0[k] = v
T2 Cache rotation → Gen1[k] = v, Gen0[k] = ∅
T3 Delete(k) applies → strict Del sees Gen0∅ → PANIC
The fix relies on two orthogonal guardrails already in force on
release/v3.0:
1. Admission-side CacheUnreachable rejects any proposal predicting
2+ rotations between propose and apply (plan.ErrCacheHorizonExceeded,
gRPC Unavailable / HTTP 503). This bounds the propose→apply race
to at most one rotation, so any key admission observed anywhere in
cache is still reachable via Gen1 at Preload.
2. The coverage gate (invariant #9) binds every FSM cache read to
admission's declared preload set — a Declare entry can only be
read by an order that emitted it.
Given those bounds, Preload-time MirrorTouch is a complete substitute
for per-intent verification: for every Declare plan entry, promote any
Gen1 hit into Gen0 before the handler runs. Applied to T3 above, Gen0
gets seeded with v before strict Del runs and the panic is gone.
Semantic changes on the release/v3.0 baseline:
- AttributeCache.Get is gen0-strict — no gen1 fallback. MirrorTouch has
already promoted anything that exists anywhere in cache before the
handler reads.
- AttributeCache.Del returns an error (kv.KV signature updated) and
refuses to fabricate a tombstone from Gen1. Preload-contract
violations bubble up loudly per invariant #7 instead of silently
desyncing.
- writeCacheTombstone writes only the current gen0 byte in 0xFF (the
gen1 byte is left as harmless stale data, purged on the next
rotation) — keeping cache mem/disk equal for the same applied index
(invariant #1).
- protoSnapshotSlot.MirrorTouch is a silent no-op when neither gen has
the key (previous behavior: touch_noop panic). Under the coverage
gate that state is legitimate.
- Resolver: CacheGuaranteed and CacheNeedsTouch both emit Declare now;
CacheMiss + Pebble-load-hit still emits Value(v). The proto's Touch
variant is retained for wire compat with pre-refactor plans still in
flight during a rolling upgrade (FSM Preload treats it identically to
Declare via MirrorTouch).
Docs: docs/technical/architecture/core/plan-intent-verification.md
walks the Path A race, the two guardrails, and the strict-Del site
checklist.
Layered on top of PR #1475 (Needs generic dispatch refactor).
gfyrag
pushed a commit
that referenced
this pull request
Jul 2, 2026
…as promote primitive
Fixes a strict AttributeCache.Del contract violation on Delete-like
orders when a concurrent Save + rotation runs between propose and apply:
T0 Admission: CheckCache(k) → CacheMiss → Declare (no seed)
T1 Concurrent Save(k, v) → Gen0[k] = v
T2 Cache rotation → Gen1[k] = v, Gen0[k] = ∅
T3 Delete(k) applies → strict Del sees Gen0∅ → PANIC
The fix relies on two orthogonal guardrails already in force on
release/v3.0:
1. Admission-side CacheUnreachable rejects any proposal predicting
2+ rotations between propose and apply (plan.ErrCacheHorizonExceeded,
gRPC Unavailable / HTTP 503). This bounds the propose→apply race
to at most one rotation, so any key admission observed anywhere in
cache is still reachable via Gen1 at Preload.
2. The coverage gate (invariant #9) binds every FSM cache read to
admission's declared preload set — a Declare entry can only be
read by an order that emitted it.
Given those bounds, Preload-time MirrorTouch is a complete substitute
for per-intent verification: for every Declare plan entry, promote any
Gen1 hit into Gen0 before the handler runs. Applied to T3 above, Gen0
gets seeded with v before strict Del runs and the panic is gone.
Semantic changes on the release/v3.0 baseline:
- AttributeCache.Get is gen0-strict — no gen1 fallback. MirrorTouch has
already promoted anything that exists anywhere in cache before the
handler reads.
- AttributeCache.Del returns an error (kv.KV signature updated) and
refuses to fabricate a tombstone from Gen1. Preload-contract
violations bubble up loudly per invariant #7 instead of silently
desyncing.
- writeCacheTombstone writes only the current gen0 byte in 0xFF (the
gen1 byte is left as harmless stale data, purged on the next
rotation) — keeping cache mem/disk equal for the same applied index
(invariant #1).
- protoSnapshotSlot.MirrorTouch is a silent no-op when neither gen has
the key (previous behavior: touch_noop panic). Under the coverage
gate that state is legitimate.
- Resolver: CacheGuaranteed and CacheNeedsTouch both emit Declare now;
CacheMiss + Pebble-load-hit still emits Value(v). The proto's Touch
variant is retained for wire compat with pre-refactor plans still in
flight during a rolling upgrade (FSM Preload treats it identically to
Declare via MirrorTouch).
Docs: docs/technical/architecture/core/plan-intent-verification.md
walks the Path A race, the two guardrails, and the strict-Del site
checklist.
Layered on top of PR #1475 (Needs generic dispatch refactor).
gfyrag
pushed a commit
that referenced
this pull request
Jul 2, 2026
…as promote primitive
Fixes a strict AttributeCache.Del contract violation on Delete-like
orders when a concurrent Save + rotation runs between propose and apply:
T0 Admission: CheckCache(k) → CacheMiss → Declare (no seed)
T1 Concurrent Save(k, v) → Gen0[k] = v
T2 Cache rotation → Gen1[k] = v, Gen0[k] = ∅
T3 Delete(k) applies → strict Del sees Gen0∅ → PANIC
The fix relies on two orthogonal guardrails already in force on
release/v3.0:
1. Admission-side CacheUnreachable rejects any proposal predicting
2+ rotations between propose and apply (plan.ErrCacheHorizonExceeded,
gRPC Unavailable / HTTP 503). This bounds the propose→apply race
to at most one rotation, so any key admission observed anywhere in
cache is still reachable via Gen1 at Preload.
2. The coverage gate (invariant #9) binds every FSM cache read to
admission's declared preload set — a Declare entry can only be
read by an order that emitted it.
Given those bounds, Preload-time MirrorTouch is a complete substitute
for per-intent verification: for every Declare plan entry, promote any
Gen1 hit into Gen0 before the handler runs. Applied to T3 above, Gen0
gets seeded with v before strict Del runs and the panic is gone.
Semantic changes on the release/v3.0 baseline:
- AttributeCache.Get is gen0-strict — no gen1 fallback. MirrorTouch has
already promoted anything that exists anywhere in cache before the
handler reads.
- AttributeCache.Del returns an error (kv.KV signature updated) and
refuses to fabricate a tombstone from Gen1. Preload-contract
violations bubble up loudly per invariant #7 instead of silently
desyncing.
- writeCacheTombstone writes only the current gen0 byte in 0xFF (the
gen1 byte is left as harmless stale data, purged on the next
rotation) — keeping cache mem/disk equal for the same applied index
(invariant #1).
- protoSnapshotSlot.MirrorTouch is a silent no-op when neither gen has
the key (previous behavior: touch_noop panic). Under the coverage
gate that state is legitimate.
- Resolver: CacheGuaranteed and CacheNeedsTouch both emit Declare now;
CacheMiss + Pebble-load-hit still emits Value(v). The proto's Touch
variant is retained for wire compat with pre-refactor plans still in
flight during a rolling upgrade (FSM Preload treats it identically to
Declare via MirrorTouch).
Docs: docs/technical/architecture/core/plan-intent-verification.md
walks the Path A race, the two guardrails, and the strict-Del site
checklist.
Layered on top of PR #1475 (Needs generic dispatch refactor).
gfyrag
pushed a commit
that referenced
this pull request
Jul 2, 2026
Replaces the systematic MirrorTouch-at-Preload pass with two in-place
primitives on AttributeCache:
* Get: gen0 → gen1 fallback. Safe under the coverage_bits gate
(invariant #9) — the fallback can only surface keys the proposer
explicitly declared.
* Del: tombstones in Gen0 when Gen0 hits; otherwise lazy-fabricates
a Gen0 tombstone borrowing Gen1's tag. Gen1's live row stays
untouched (shadowed by the Gen0 tombstone, purged on the next
rotation). writeCacheTombstone still writes a single row to the
Gen0 byte — memory equals disk (invariant #1).
Preload skips coverage-only AttributeCoverage entries entirely (only
seed intents call MirrorPreload). The MirrorTouch method / plumbing
and the AttributeCache.Touch / Cache.TouchByType helpers are deleted
as dead code. The CacheHit verdict collapses "already in Gen0" and
"Gen1-only" into a single admission signal; the FSM's read horizon is
still bounded to admission's declared preload set by coverage_bits.
CacheUnreachable (rejects proposals with ≥2 predicted rotations)
keeps the propose→apply race window bounded to at most one rotation,
which is what the fallback + lazy fabrication rely on.
Doc: docs/technical/architecture/core/plan-intent-verification.md
Regression: internal/infra/cache/keystore_delete_test.go +
internal/infra/state/cache_snapshotter_test.go
(TestCacheSnapshotter_EN1242_DeleteAfterRotationCrashRestart)
gfyrag
pushed a commit
that referenced
this pull request
Jul 2, 2026
Replaces the systematic MirrorTouch-at-Preload pass with two in-place
primitives on AttributeCache:
* Get: gen0 → gen1 fallback. Safe under the coverage_bits gate
(invariant #9) — the fallback can only surface keys the proposer
explicitly declared.
* Del: tombstones in Gen0 when Gen0 hits; otherwise lazy-fabricates
a Gen0 tombstone borrowing Gen1's tag. Gen1's live row stays
untouched (shadowed by the Gen0 tombstone, purged on the next
rotation). writeCacheTombstone still writes a single row to the
Gen0 byte — memory equals disk (invariant #1).
Preload skips coverage-only AttributeCoverage entries entirely (only
seed intents call MirrorPreload). The MirrorTouch method / plumbing
and the AttributeCache.Touch / Cache.TouchByType helpers are deleted
as dead code. The CacheHit verdict collapses "already in Gen0" and
"Gen1-only" into a single admission signal; the FSM's read horizon is
still bounded to admission's declared preload set by coverage_bits.
CacheUnreachable (rejects proposals with ≥2 predicted rotations)
keeps the propose→apply race window bounded to at most one rotation,
which is what the fallback + lazy fabrication rely on.
Doc: docs/technical/architecture/core/plan-intent-verification.md
Regression: internal/infra/cache/keystore_delete_test.go +
internal/infra/state/cache_snapshotter_test.go
(TestCacheSnapshotter_EN1242_DeleteAfterRotationCrashRestart)
gfyrag
pushed a commit
that referenced
this pull request
Jul 3, 2026
Replaces the systematic MirrorTouch-at-Preload pass with two in-place
primitives on AttributeCache:
* Get: gen0 → gen1 fallback. Safe under the coverage_bits gate
(invariant #9) — the fallback can only surface keys the proposer
explicitly declared.
* Del: tombstones in Gen0 when Gen0 hits; otherwise lazy-fabricates
a Gen0 tombstone borrowing Gen1's tag. Gen1's live row stays
untouched (shadowed by the Gen0 tombstone, purged on the next
rotation). writeCacheTombstone still writes a single row to the
Gen0 byte — memory equals disk (invariant #1).
Preload skips coverage-only AttributeCoverage entries entirely (only
seed intents call MirrorPreload). The MirrorTouch method / plumbing
and the AttributeCache.Touch / Cache.TouchByType helpers are deleted
as dead code. The CacheHit verdict collapses "already in Gen0" and
"Gen1-only" into a single admission signal; the FSM's read horizon is
still bounded to admission's declared preload set by coverage_bits.
CacheUnreachable (rejects proposals with ≥2 predicted rotations)
keeps the propose→apply race window bounded to at most one rotation,
which is what the fallback + lazy fabrication rely on.
Doc: docs/technical/architecture/core/plan-intent-verification.md
Regression: internal/infra/cache/keystore_delete_test.go +
internal/infra/state/cache_snapshotter_test.go
(TestCacheSnapshotter_EN1242_DeleteAfterRotationCrashRestart)
gfyrag
pushed a commit
that referenced
this pull request
Jul 3, 2026
Replaces the systematic MirrorTouch-at-Preload pass with two in-place
primitives on AttributeCache:
* Get: gen0 → gen1 fallback. Safe under the coverage_bits gate
(invariant #9) — the fallback can only surface keys the proposer
explicitly declared.
* Del: tombstones in Gen0 when Gen0 hits; otherwise lazy-fabricates
a Gen0 tombstone borrowing Gen1's tag. Gen1's live row stays
untouched (shadowed by the Gen0 tombstone, purged on the next
rotation). writeCacheTombstone still writes a single row to the
Gen0 byte — memory equals disk (invariant #1).
Preload skips coverage-only AttributeCoverage entries entirely (only
seed intents call MirrorPreload). The MirrorTouch method / plumbing
and the AttributeCache.Touch / Cache.TouchByType helpers are deleted
as dead code. The CacheHit verdict collapses "already in Gen0" and
"Gen1-only" into a single admission signal; the FSM's read horizon is
still bounded to admission's declared preload set by coverage_bits.
CacheUnreachable (rejects proposals with ≥2 predicted rotations)
keeps the propose→apply race window bounded to at most one rotation,
which is what the fallback + lazy fabrication rely on.
Doc: docs/technical/architecture/core/plan-intent-verification.md
Regression: internal/infra/cache/keystore_delete_test.go +
internal/infra/state/cache_snapshotter_test.go
(TestCacheSnapshotter_EN1242_DeleteAfterRotationCrashRestart)
gfyrag
pushed a commit
that referenced
this pull request
Jul 3, 2026
Replaces the systematic MirrorTouch-at-Preload pass with two in-place
primitives on AttributeCache:
* Get: gen0 → gen1 fallback. Safe under the coverage_bits gate
(invariant #9) — the fallback can only surface keys the proposer
explicitly declared.
* Del: tombstones in Gen0 when Gen0 hits; otherwise lazy-fabricates
a Gen0 tombstone borrowing Gen1's tag. Gen1's live row stays
untouched (shadowed by the Gen0 tombstone, purged on the next
rotation). writeCacheTombstone still writes a single row to the
Gen0 byte — memory equals disk (invariant #1).
Preload skips coverage-only AttributeCoverage entries entirely (only
seed intents call MirrorPreload). The MirrorTouch method / plumbing
and the AttributeCache.Touch / Cache.TouchByType helpers are deleted
as dead code. The CacheHit verdict collapses "already in Gen0" and
"Gen1-only" into a single admission signal; the FSM's read horizon is
still bounded to admission's declared preload set by coverage_bits.
CacheUnreachable (rejects proposals with ≥2 predicted rotations)
keeps the propose→apply race window bounded to at most one rotation,
which is what the fallback + lazy fabrication rely on.
Doc: docs/technical/architecture/core/plan-intent-verification.md
Regression: internal/infra/cache/keystore_delete_test.go +
internal/infra/state/cache_snapshotter_test.go
(TestCacheSnapshotter_EN1242_DeleteAfterRotationCrashRestart)
gfyrag
pushed a commit
that referenced
this pull request
Jul 3, 2026
Replaces the systematic MirrorTouch-at-Preload pass with two in-place
primitives on AttributeCache:
* Get: gen0 → gen1 fallback. Safe under the coverage_bits gate
(invariant #9) — the fallback can only surface keys the proposer
explicitly declared.
* Del: tombstones in Gen0 when Gen0 hits; otherwise lazy-fabricates
a Gen0 tombstone borrowing Gen1's tag. Gen1's live row stays
untouched (shadowed by the Gen0 tombstone, purged on the next
rotation). writeCacheTombstone still writes a single row to the
Gen0 byte — memory equals disk (invariant #1).
Preload skips coverage-only AttributeCoverage entries entirely (only
seed intents call MirrorPreload). The MirrorTouch method / plumbing
and the AttributeCache.Touch / Cache.TouchByType helpers are deleted
as dead code. The CacheHit verdict collapses "already in Gen0" and
"Gen1-only" into a single admission signal; the FSM's read horizon is
still bounded to admission's declared preload set by coverage_bits.
CacheUnreachable (rejects proposals with ≥2 predicted rotations)
keeps the propose→apply race window bounded to at most one rotation,
which is what the fallback + lazy fabrication rely on.
Doc: docs/technical/architecture/core/plan-intent-verification.md
Regression: internal/infra/cache/keystore_delete_test.go +
internal/infra/state/cache_snapshotter_test.go
(TestCacheSnapshotter_EN1242_DeleteAfterRotationCrashRestart)
gfyrag
pushed a commit
that referenced
this pull request
Jul 3, 2026
Replaces the systematic MirrorTouch-at-Preload pass with two in-place
primitives on AttributeCache:
* Get: gen0 → gen1 fallback. Safe under the coverage_bits gate
(invariant #9) — the fallback can only surface keys the proposer
explicitly declared.
* Del: tombstones in Gen0 when Gen0 hits; otherwise lazy-fabricates
a Gen0 tombstone borrowing Gen1's tag. Gen1's live row stays
untouched (shadowed by the Gen0 tombstone, purged on the next
rotation). writeCacheTombstone still writes a single row to the
Gen0 byte — memory equals disk (invariant #1).
Preload skips coverage-only AttributeCoverage entries entirely (only
seed intents call MirrorPreload). The MirrorTouch method / plumbing
and the AttributeCache.Touch / Cache.TouchByType helpers are deleted
as dead code. The CacheHit verdict collapses "already in Gen0" and
"Gen1-only" into a single admission signal; the FSM's read horizon is
still bounded to admission's declared preload set by coverage_bits.
CacheUnreachable (rejects proposals with ≥2 predicted rotations)
keeps the propose→apply race window bounded to at most one rotation,
which is what the fallback + lazy fabrication rely on.
Doc: docs/technical/architecture/core/plan-intent-verification.md
Regression: internal/infra/cache/keystore_delete_test.go +
internal/infra/state/cache_snapshotter_test.go
(TestCacheSnapshotter_EN1242_DeleteAfterRotationCrashRestart)
gfyrag
added a commit
that referenced
this pull request
Jul 3, 2026
Replaces the systematic MirrorTouch-at-Preload pass with two in-place
primitives on AttributeCache:
* Get: gen0 → gen1 fallback. Safe under the coverage_bits gate
(invariant #9) — the fallback can only surface keys the proposer
explicitly declared.
* Del: tombstones in Gen0 when Gen0 hits; otherwise lazy-fabricates
a Gen0 tombstone borrowing Gen1's tag. Gen1's live row stays
untouched (shadowed by the Gen0 tombstone, purged on the next
rotation). writeCacheTombstone still writes a single row to the
Gen0 byte — memory equals disk (invariant #1).
Preload skips coverage-only AttributeCoverage entries entirely (only
seed intents call MirrorPreload). The MirrorTouch method / plumbing
and the AttributeCache.Touch / Cache.TouchByType helpers are deleted
as dead code. The CacheHit verdict collapses "already in Gen0" and
"Gen1-only" into a single admission signal; the FSM's read horizon is
still bounded to admission's declared preload set by coverage_bits.
CacheUnreachable (rejects proposals with ≥2 predicted rotations)
keeps the propose→apply race window bounded to at most one rotation,
which is what the fallback + lazy fabrication rely on.
Doc: docs/technical/architecture/core/plan-intent-verification.md
Regression: internal/infra/cache/keystore_delete_test.go +
internal/infra/state/cache_snapshotter_test.go
(TestCacheSnapshotter_EN1242_DeleteAfterRotationCrashRestart)
Co-authored-by: Geoffrey Ragot <geoffrey@formance.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.