Skip to content

offload: fingerprint-verified vector upgrades + whole-state pending gate#178

Merged
mbertschler merged 2 commits into
claude/issue-154-verify-durability-cadencesfrom
claude/issue-155-fingerprint-verified
Jul 24, 2026
Merged

offload: fingerprint-verified vector upgrades + whole-state pending gate#178
mbertschler merged 2 commits into
claude/issue-154-verify-durability-cadencesfrom
claude/issue-155-fingerprint-verified

Conversation

@mbertschler

Copy link
Copy Markdown
Owner

Summary

Makes offload reachable against packed / content-addressed offsites by fixing the two friction-log defects (F13 + F29) that together closed every gate path in the reference household. Implements Martin's option 1b.

Stacked on #154 (PR #172); base retargets to main once #154 merges. The diff shown here is only this issue's delta.

Changes

F13 — the pending-artifact gate was per-run. packedHandler.certifyPacked held the vector only when this run's packs were unverified, so a later run that packed nothing advanced the whole volume past an earlier still-pending pack, and once squirrel verify filled the fingerprints nothing re-advanced. The advance is now gated on zero pending artifacts for the whole (volume, destination) pair (store.CountVolumeContentsPendingFingerprint), and squirrel verify re-attempts the advance itself after a clean pass (store.UpgradeDestinationVectorToFingerprintVerified, iterated over every volume — a no-op for a volume never pushed to the destination or still carrying a pending artifact).

F29 — the component method was never upgraded. Packed/CA components stayed presence+size forever, so the fail-closed gate refused relayed offsites. Capture (both handlers) and verify now upgrade a fully fingerprint-verified pair to a new fingerprint-verified verify_method. It is a free-text value — no schema migration, no SchemaVersion bump (the column has no CHECK constraint). It relays over the durability pull like any other method.

Decision 1b — cadence coupling. A verified provider-fingerprint chain counts as content-verified for the gate, but only while a scheduled verify cadence (#154) keeps re-confirming it:

  • The responder relays its effective verify cadence per destination (DurabilityComponent.VerifyEveryNs, mirroring how VerifiedAtNs relays).
  • The pull bakes a relayed fingerprint-verified component down to presence+size when that cadence is absent (older peer, or a destination the responder runs on no cadence) — fail-closed.
  • The gate accepts fingerprint-verified when relayed (cadence proven at pull time) or when the local destination has a live cadence; a local component without a cadence falls back to the per-object/pack scan-back, since the node re-reads its own fingerprints. The coupling binds the relayed path, where the evidence cannot be re-checked locally.

syncproto addition is version-gated: absent on older peers → zero → fail closed. KnownVerifyMethod accepts the new method at the wire boundary; ContentVerifiedMethod deliberately does not (its acceptance is cadence-coupled, applied by the gate).

Testing

  • go vet ./... — clean
  • go test ./... — all pass
  • golangci-lint run (v2.5.0 rebuilt with GOTOOLCHAIN=go1.26.1 due to a toolchain mismatch) — 0 issues

New tests: both F13 sides (per-state hold; verify-then-advance) for packed and content-addressed, the capture-time method upgrade, the relayed accept / relayed-baked-down refuse cadence coupling, the local cadence/fallback/refuse cases, the pull-time baking (relayedMethod), and the responder relaying its cadence.

Closes #155

Stacked on #154 (PR #172); base retargets to main once #154 merges.

🤖 Generated with Claude Code

https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7


Generated by Claude Code

Make offload reachable against packed/CA offsites by fixing the two
friction-log defects that together closed every gate path (F13 + F29).

F13 — the pending-artifact gate was per-run. certifyPacked held the
vector only when *this run's* packs were unverified, so a later run that
packed nothing advanced the whole volume past an earlier still-pending
pack, and once `squirrel verify` filled the fingerprints nothing
re-advanced. The advance is now gated on zero pending artifacts for the
whole (volume, destination) pair (CountVolumeContentsPendingFingerprint),
and `squirrel verify` re-attempts the advance itself once a clean pass
completes (UpgradeDestinationVectorToFingerprintVerified).

F29 — the component method was never upgraded past presence+size, so the
fail-closed gate refused relayed offsites forever. Capture and verify now
upgrade a fully fingerprint-verified pair to a new `fingerprint-verified`
verify_method (a free-text value; no schema change, no CHECK constraint).
It relays over the durability pull like any other method.

Decision 1b — a verified provider-fingerprint chain counts as
content-verified for the gate, but only while a scheduled verify cadence
(#154) keeps re-confirming it. The responder relays its effective verify
cadence per destination (DurabilityComponent.VerifyEveryNs); the pull
bakes a relayed fingerprint-verified component down to presence+size when
that cadence is absent (older peer, or a destination the responder runs
on no cadence), failing closed. The gate accepts a fingerprint-verified
component when relayed (cadence proven at pull time) or when the local
destination carries a live cadence; a local component without a cadence
falls back to the per-object/pack scan-back, since the node re-reads its
own fingerprints — the coupling binds the relayed path, where the
evidence cannot be re-checked locally.

Tests: both F13 sides (per-state hold, verify-then-advance) for packed
and content-addressed, the capture-time method upgrade, the relayed
accept / relayed-absent refuse cadence coupling, the local
cadence/fallback/refuse cases, the pull-time baking, and the responder
relaying its cadence.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
Copilot AI review requested due to automatic review settings July 24, 2026 11:55

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes offload reachable for packed and content-addressed offsites by upgrading durability evidence to a new fingerprint-verified verify method once provider fingerprints are fully confirmed, and by gating vector advancement on the whole (volume, destination) pending-fingerprint state (not just the current run). It also cadence-couples relayed fingerprint-verified evidence by relaying the responder’s effective verify cadence and baking the method down when that cadence is absent.

Changes:

  • Add fingerprint-verified as a new durability verify method; upgrade vectors to it when all present content for a (volume, destination) is fingerprint-verified (including verify-triggered upgrades).
  • Relay responder verify cadence (verify_every_ns) in durability responses; on pull, bake down fingerprint-verified to presence+size when cadence is missing (fail-closed).
  • Update offload gate and CLI wiring to accept fingerprint-verified only when cadence-coupled (relayed) or locally cadenced; add regression tests for F13/F29 and coupling behavior.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
syncproto/syncproto.go Adds verify_every_ns to durability components so cadence can be relayed.
sync/verify_remote.go After a clean verify pass, upgrades destination vectors to fingerprint-verified where eligible.
sync/packed.go Switches packed certification gating to whole-state pending-fingerprint count and stamps vector as fingerprint-verified.
sync/packed_test.go Adds packed regression tests for verify-then-advance and per-state pending gate behavior.
sync/handler.go Exposes VerifyMethodFingerprint constant alias for handler/report semantics.
sync/durability.go Bakes down pulled fingerprint-verified evidence when responder cadence is absent.
sync/durability_test.go Tests cadence-coupled baking behavior for relayed methods.
sync/content_addressed.go Chooses advance method (presence+size vs fingerprint-verified) based on whole-state pending fingerprints.
sync/content_addressed_test.go Adds content-addressed regression test for verify-triggered vector upgrade.
store/destination_run_ids.go Introduces VerifyMethodFingerprint, upgrades known-method validation, and adds vector upgrade helper.
store/destination_run_ids_test.go Updates content-verified method expectations to exclude fingerprint-verified from unconditional acceptance.
store/content_presence.go Adds whole-state pending-fingerprint counter used by capture/verify upgrade gates.
offload/offload.go Threads local cadence knowledge into gate loading.
offload/gate.go Adds cadence-coupled acceptance for fingerprint-verified and fallback to scan-back when not cadenced.
offload/durability_soundness_test.go Adds offload tests for relayed/local fingerprint-verified cadence coupling behavior.
cmd/squirrel/offload.go Computes per-required-target local cadence presence and passes it into offload options.
agent/durability.go Relays effective verify cadence per destination in durability responses.
agent/durability_test.go Tests verify cadence relay behavior on the durability endpoint.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sync/packed.go Outdated
The comment implied the fingerprint pending/advance gate covers the
placement map and manifest segment, but CountVolumeContentsPendingFingerprint
only considers the volume's present file contents (verified via
remote_objects / remote_packs). Reword to state exactly what the gate
covers — packs and per-hash objects, the artifacts that carry content bytes
— and note that the placement map and manifest segment are re-derivable
metadata whose landing is confirmed upstream in push (presence+size, before
the run is promoted to success), so a run whose map or segment failed to
land never reaches certifyPacked. Comment-only; no behaviour change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
@mbertschler
mbertschler merged commit dbae8e1 into claude/issue-154-verify-durability-cadences Jul 24, 2026
3 checks passed
@mbertschler
mbertschler deleted the claude/issue-155-fingerprint-verified branch July 24, 2026 21:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants