Add spec-declared checksums for static tables; validate lockfile fast path - #32
Merged
Conversation
… path Builds on the download-integrity work: sidecar hashes are computed from whatever was downloaded, so they cannot catch a cache that was poisoned before the sidecar was written, and the ResolvePipeline lockfile fast path trusted entries with only an existence check — its 'will re-resolve on next run' log line was never true, since the fast path always returned. - ResourceProductSpec/SearchTarget gain an optional checksum field (sha256:<hex> of the file as served), carried through ResourceCatalog.build. Meaningful for entries that resolve to exactly one file from an immutable source. - WormHole.download_one treats a declared checksum as authoritative: cache hits must match it (evict + re-download otherwise, even when the sidecar agrees with the corrupt content), and fresh downloads are verified against it with one retry before anything is cached. - ResolvePipeline validates lockfile entries against their per-file sidecar hashes (aggregate entries carry no hash of their own) and falls through to full re-resolution when any required entry is missing or corrupt, overwriting the stale lockfile. - pride_table_config.yaml declares checksums for all 12 table products, computed from the files at the pinned PRIDE-PPPAR commit b7451a8 and cross-checked against the GitHub Contents API sizes. The re-pin procedure note now includes regenerating them. Part of #26 (phase 3 of 3). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 3 of 3 for #26. Re-opened as a fresh PR: this was #31, which GitHub auto-closed when its stacked base branch (#29's) was deleted on merge. Identical content, now based on
main; CI already passed on this commit in #31.Phase 2's (#29) sidecar hashes are computed from whatever was downloaded, so they cannot catch a cache poisoned before the sidecar was written. And the
ResolvePipelinelockfile fast path — the mechanism that made a truncated table "resolved as satisfied on every later run" — trusted entries with only an existence check; its "will re-resolve on next run" log line was never true, since the fast path always returned.Changes
Spec-declared checksums.
ResourceProductSpecandSearchTargetgain an optionalchecksumfield (sha256:<hex>of the file as served), carried throughResourceCatalog.build. Intended for entries that resolve to exactly one file from an immutable source.WormHole treats a declared checksum as authoritative. Cache hits must match it — evicted and re-downloaded otherwise, even when the sidecar agrees with the corrupt content. Fresh downloads are verified against it (after the size check) with one retry before anything is cached.
Lockfile fast-path validation.
ResolvePipelinenow validates each aggregate-lockfile entry against its per-file sidecar hash (aggregate entries carry no hash of their own), and when any required entry is missing or corrupt it falls through to a full re-resolution, overwriting the stale lockfile — making the existing log message true.All 12 PRIDE table products declare checksums in
pride_table_config.yaml, computed from the files at the pinned commitb7451a8(Pin PRIDE-PPPAR table source to commit SHA instead of floating master #27) and cross-checked against the GitHub Contents API sizes. The re-pin procedure comment now includes regenerating them.Verification
ResourceCatalogpass-through, and lockfile-entry validation. 430 non-integration tests pass; ruff check/format clean.pride_table_config.yamlparses viaResourceSpec.from_yamlwith all 12 checksums present.Known limitation
A file found by local search (not via lockfile or download) is still trusted without a checksum comparison. With downloads verified at creation and the fast path validated, no new poisoned cache can be created; this residual applies only to pre-existing corrupt files placed by other means.
Closes #26.
🤖 Generated with Claude Code