feat: emit and seed the evo snapshot section in dumptxoutset and snapshot load - #7601
Draft
PastaPastaPasta wants to merge 5 commits into
Draft
feat: emit and seed the evo snapshot section in dumptxoutset and snapshot load#7601PastaPastaPasta wants to merge 5 commits into
PastaPastaPasta wants to merge 5 commits into
Conversation
…e validation First code PR of the assumeutxo M4 series (dashpay#7579 decomposition): the versioned interchange format for Dash's evo state alongside a UTXO snapshot - canonical serialization, DoS-bounded validating decode, and every validation invariant that needs no chain context. Chain-aware building/validation and dump/load integration follow in the next PRs of the series. Canonical ordering exists because snapshot content is hashed and cross-checked; per-object serializers are reused through a bounded stream wrapper, with bespoke code only at container level (ordering, bounds, per-entry budgets); decode-time checks deliberately stay out of the trusted hot EvoDB deserializers. AssumeutxoData gains the EvoSnapshotHash anchor the format is pinned by. Includes the aggregate rotation skip-list bound (lists accumulate across every quorum index and wrap the combined MN list), the CRangesSet bounded unserializer, and a vendored-immer shift-base ubsan suppression reachable only through the deliberately hash-colliding test fixtures. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… sign change The mask for rejecting out-of-range trailing bits promotes through operator~ to a negative int before its implicit conversion back to uint8_t, which clang's implicit-integer-sign-change check reports for every bitset whose size is not a multiple of eight. The evo snapshot unit tests are the first to deserialize such bitsets under the sanitizer job. Same bits, stated explicitly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Second code PR of the assumeutxo M4 series: BuildEvoSnapshot() collects commitments, rotation snapshots, historical MN-list diffs, and modifiers from chain state; ValidateEvoSnapshotAgainstChain() verifies a decoded snapshot against the block index, deployments, and quorum reconstruction; and the EvoDB seeding surface (canonical MN lists, mined commitments, credit pool, MNHF signals, quorum snapshots and modifiers) writes exactly the state reconstruction later reads. GetHashModifier() prefers a seeded exact modifier when the work block's data is unavailable and cross-checks it against recomputation when it is; a mismatch throws SnapshotStateMismatchError, unreachable in production until the load-time integration seeds modifiers, at which point the next PR routes it into the controlled invalid-snapshot path. GetDeterministicMNListHash() switches to the canonical codec hash so completion markers and payload-derived hashes share an identity; nothing on develop compares the old and new encodings across an upgrade except a dev-channel node mid-snapshot-validation, which would fail completion conservatively. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…izon BuildEvoSnapshot() required every enabled LLMQ type to have exactly signingActiveQuorumCount active commitments plus the complete safety and rotation horizons, and both validation layers enforced the same equalities. A young chain, a freshly activated quorum type, or a rotation type whose first cycles predate its activation cannot satisfy that, so dumptxoutset would fail on perfectly valid chain state. Reported by review on the predecessor PR. The builder now emits the history that exists (clamped to the parameter-derived maxima), context-free validation treats the parameter counts as upper bounds, and chain-aware validation requires every carried entry to sit at a derived horizon position while tolerating absent ones. Withholding available history is not made easier by this: the completion-time CbTx quorum merkle root pins the active set exactly, historical-diff and modifier tallies must still mirror the carried commitments, and quorum reconstruction against chain state establishes sufficiency at completion. Two regression tests pin the young-chain builder path and the partial-history validation path; both fail against the previous exact-count enforcement. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…shot load Final PR of the assumeutxo M4 series: dumptxoutset appends the canonical evo section (marker, version, payload, hash) after the UTXO records; snapshot activation decodes it, validates it context-free and against the chain, seeds EvoDB through the seeding surface, retains the section for the deferred completion-time CbTx cross-check, and records background MN-list hashes only for the snapshot base and the bounded set of historical work blocks. The completion path cross-checks reconstructed historical MN lists, verifies the retained section against the base CbTx, and routes runtime seeded-state mismatches (GetAllQuorumMembers) into the controlled invalid-snapshot shutdown. The assumeutxo prune lock keeps the base block's data available until that check completes. Per review on the original M4 PR, every ChainstateManager body stays in validation.cpp: the previously proposed evo/snapshot_load.cpp move is gone, with the chain-facing helpers declared in evo/snapshot.h and the load/build/completion logic living where upstream keeps it. feature_assumeutxo_dash.py exercises the dump/load/complete cycle end to end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue being fixed or feature implemented
Stacked on #7592 and #7593 — the first commits shown here belong to those PRs; this one adds the final commit. Final PR of the AssumeUTXO M4 decomposition (#7579 — series map there): the lifecycle wiring that makes the format real.
dumptxoutsetemits the canonical evo section alongside the UTXO set; snapshot load decodes, validates, and seeds it; completion cross-checks everything the earlier PRs promised (retained-section CbTx verification, historical MN-list reconstruction, seeded-modifier integrity) and the assumeutxo prune lock keeps the base block readable until that happens.What was done?
dumptxoutsetappends the marker/version/payload/hash evo section; load reads it (AutoFile::size()bounds the decode), validates context-free and against chain, and seeds EvoDB through the feat: build, chain-validate, and seed evo snapshot state #7593 seeding surface without publishing into shared caches. The section is retained in EvoDB (EVODB_SNAPSHOT_EVO_SECTION) for the deferred completion-time CbTx cross-check.GetAllQuorumMembers) route into the controlled invalid-snapshot shutdown (HandleSnapshotStateMismatch), completing the plumbing feat: build, chain-validate, and seed evo snapshot state #7593 deliberately left unwired.ProtectSnapshotBaseFromPruning/release,BlockManager::DeletePruneLock) — the remainder of the original B6 commit, placed with its consumer as discussed in backport: assumeutxo M4 — evo snapshot format v3 and LLMQ reconstruction #7579 review.ChainstateManagerbody stays invalidation.cpp— the previously proposedevo/snapshot_load.cppfile is gone from the series; chain-facing helpers are declared inevo/snapshot.hand implemented where upstream keeps the logic.feature_assumeutxo_dash.py(dump → load → background-validate → complete, plus recovery/invalid paths),rpc_dumptxoutset.pyupdate, and the unit-test deltas for the lifecycle (prune-lock survival, soft-fail base detection, EvoDB retention).How Has This Been Tested?
Full unit suite green on a
--enable-werrorbuild;feature_assumeutxo_dash.py,rpc_dumptxoutset.py, andfeature_reindex.pypass locally. The stack beneath it carries its own sanitizer verification.Breaking Changes
None.
dumptxoutsetoutput gains the evo section (new format version); old snapshots without it are rejected at load on DIP3-active chains, which is the intended security posture — there is no legacy Dash snapshot format in the wild.Checklist: