offload: fingerprint-verified vector upgrades + whole-state pending gate#178
Merged
mbertschler merged 2 commits intoJul 24, 2026
Conversation
Make offload reachable against packed/CA offsites by fixing the two friction-log defects that together closed every gate path (F13 + F29). F13 — the pending-artifact gate was per-run. certifyPacked held the vector only when *this run's* packs were unverified, so a later run that packed nothing advanced the whole volume past an earlier still-pending pack, and once `squirrel verify` filled the fingerprints nothing re-advanced. The advance is now gated on zero pending artifacts for the whole (volume, destination) pair (CountVolumeContentsPendingFingerprint), and `squirrel verify` re-attempts the advance itself once a clean pass completes (UpgradeDestinationVectorToFingerprintVerified). F29 — the component method was never upgraded past presence+size, so the fail-closed gate refused relayed offsites forever. Capture and verify now upgrade a fully fingerprint-verified pair to a new `fingerprint-verified` verify_method (a free-text value; no schema change, no CHECK constraint). It relays over the durability pull like any other method. Decision 1b — a verified provider-fingerprint chain counts as content-verified for the gate, but only while a scheduled verify cadence (#154) keeps re-confirming it. The responder relays its effective verify cadence per destination (DurabilityComponent.VerifyEveryNs); the pull bakes a relayed fingerprint-verified component down to presence+size when that cadence is absent (older peer, or a destination the responder runs on no cadence), failing closed. The gate accepts a fingerprint-verified component when relayed (cadence proven at pull time) or when the local destination carries a live cadence; a local component without a cadence falls back to the per-object/pack scan-back, since the node re-reads its own fingerprints — the coupling binds the relayed path, where the evidence cannot be re-checked locally. Tests: both F13 sides (per-state hold, verify-then-advance) for packed and content-addressed, the capture-time method upgrade, the relayed accept / relayed-absent refuse cadence coupling, the local cadence/fallback/refuse cases, the pull-time baking, and the responder relaying its cadence. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
There was a problem hiding this comment.
Pull request overview
This PR makes offload reachable for packed and content-addressed offsites by upgrading durability evidence to a new fingerprint-verified verify method once provider fingerprints are fully confirmed, and by gating vector advancement on the whole (volume, destination) pending-fingerprint state (not just the current run). It also cadence-couples relayed fingerprint-verified evidence by relaying the responder’s effective verify cadence and baking the method down when that cadence is absent.
Changes:
- Add
fingerprint-verifiedas a new durability verify method; upgrade vectors to it when all present content for a (volume, destination) is fingerprint-verified (including verify-triggered upgrades). - Relay responder verify cadence (
verify_every_ns) in durability responses; on pull, bake downfingerprint-verifiedtopresence+sizewhen cadence is missing (fail-closed). - Update offload gate and CLI wiring to accept
fingerprint-verifiedonly when cadence-coupled (relayed) or locally cadenced; add regression tests for F13/F29 and coupling behavior.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| syncproto/syncproto.go | Adds verify_every_ns to durability components so cadence can be relayed. |
| sync/verify_remote.go | After a clean verify pass, upgrades destination vectors to fingerprint-verified where eligible. |
| sync/packed.go | Switches packed certification gating to whole-state pending-fingerprint count and stamps vector as fingerprint-verified. |
| sync/packed_test.go | Adds packed regression tests for verify-then-advance and per-state pending gate behavior. |
| sync/handler.go | Exposes VerifyMethodFingerprint constant alias for handler/report semantics. |
| sync/durability.go | Bakes down pulled fingerprint-verified evidence when responder cadence is absent. |
| sync/durability_test.go | Tests cadence-coupled baking behavior for relayed methods. |
| sync/content_addressed.go | Chooses advance method (presence+size vs fingerprint-verified) based on whole-state pending fingerprints. |
| sync/content_addressed_test.go | Adds content-addressed regression test for verify-triggered vector upgrade. |
| store/destination_run_ids.go | Introduces VerifyMethodFingerprint, upgrades known-method validation, and adds vector upgrade helper. |
| store/destination_run_ids_test.go | Updates content-verified method expectations to exclude fingerprint-verified from unconditional acceptance. |
| store/content_presence.go | Adds whole-state pending-fingerprint counter used by capture/verify upgrade gates. |
| offload/offload.go | Threads local cadence knowledge into gate loading. |
| offload/gate.go | Adds cadence-coupled acceptance for fingerprint-verified and fallback to scan-back when not cadenced. |
| offload/durability_soundness_test.go | Adds offload tests for relayed/local fingerprint-verified cadence coupling behavior. |
| cmd/squirrel/offload.go | Computes per-required-target local cadence presence and passes it into offload options. |
| agent/durability.go | Relays effective verify cadence per destination in durability responses. |
| agent/durability_test.go | Tests verify cadence relay behavior on the durability endpoint. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The comment implied the fingerprint pending/advance gate covers the placement map and manifest segment, but CountVolumeContentsPendingFingerprint only considers the volume's present file contents (verified via remote_objects / remote_packs). Reword to state exactly what the gate covers — packs and per-hash objects, the artifacts that carry content bytes — and note that the placement map and manifest segment are re-derivable metadata whose landing is confirmed upstream in push (presence+size, before the run is promoted to success), so a run whose map or segment failed to land never reaches certifyPacked. Comment-only; no behaviour change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
mbertschler
merged commit Jul 24, 2026
dbae8e1
into
claude/issue-154-verify-durability-cadences
3 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes offload reachable against packed / content-addressed offsites by fixing the two friction-log defects (F13 + F29) that together closed every gate path in the reference household. Implements Martin's option 1b.
Stacked on #154 (PR #172); base retargets to
mainonce #154 merges. The diff shown here is only this issue's delta.Changes
F13 — the pending-artifact gate was per-run.
packedHandler.certifyPackedheld the vector only when this run's packs were unverified, so a later run that packed nothing advanced the whole volume past an earlier still-pending pack, and oncesquirrel verifyfilled the fingerprints nothing re-advanced. The advance is now gated on zero pending artifacts for the whole (volume, destination) pair (store.CountVolumeContentsPendingFingerprint), andsquirrel verifyre-attempts the advance itself after a clean pass (store.UpgradeDestinationVectorToFingerprintVerified, iterated over every volume — a no-op for a volume never pushed to the destination or still carrying a pending artifact).F29 — the component method was never upgraded. Packed/CA components stayed
presence+sizeforever, so the fail-closed gate refused relayed offsites. Capture (both handlers) and verify now upgrade a fully fingerprint-verified pair to a newfingerprint-verifiedverify_method. It is a free-text value — no schema migration, noSchemaVersionbump (the column has no CHECK constraint). It relays over the durability pull like any other method.Decision 1b — cadence coupling. A verified provider-fingerprint chain counts as content-verified for the gate, but only while a scheduled verify cadence (#154) keeps re-confirming it:
DurabilityComponent.VerifyEveryNs, mirroring howVerifiedAtNsrelays).fingerprint-verifiedcomponent down topresence+sizewhen that cadence is absent (older peer, or a destination the responder runs on no cadence) — fail-closed.fingerprint-verifiedwhen relayed (cadence proven at pull time) or when the local destination has a live cadence; a local component without a cadence falls back to the per-object/pack scan-back, since the node re-reads its own fingerprints. The coupling binds the relayed path, where the evidence cannot be re-checked locally.syncprotoaddition is version-gated: absent on older peers → zero → fail closed.KnownVerifyMethodaccepts the new method at the wire boundary;ContentVerifiedMethoddeliberately does not (its acceptance is cadence-coupled, applied by the gate).Testing
go vet ./...— cleango test ./...— all passgolangci-lint run(v2.5.0 rebuilt withGOTOOLCHAIN=go1.26.1due to a toolchain mismatch) — 0 issuesNew tests: both F13 sides (per-state hold; verify-then-advance) for packed and content-addressed, the capture-time method upgrade, the relayed accept / relayed-baked-down refuse cadence coupling, the local cadence/fallback/refuse cases, the pull-time baking (
relayedMethod), and the responder relaying its cadence.Closes #155
Stacked on #154 (PR #172); base retargets to main once #154 merges.
🤖 Generated with Claude Code
https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
Generated by Claude Code