Skip to content

feat: evo snapshot v3 — canonical bounded codec and context-free validation - #7592

Draft
PastaPastaPasta wants to merge 3 commits into
dashpay:developfrom
PastaPastaPasta:feat/evo-snapshot-format
Draft

feat: evo snapshot v3 — canonical bounded codec and context-free validation#7592
PastaPastaPasta wants to merge 3 commits into
dashpay:developfrom
PastaPastaPasta:feat/evo-snapshot-format

Conversation

@PastaPastaPasta

Copy link
Copy Markdown
Member

Issue being fixed or feature implemented

Part of the AssumeUTXO M4 decomposition (#7579, now draft — see the series map there). For a Dash node, a UTXO snapshot alone is not enough to operate at the base block: the node also needs the deterministic MN list, quorum commitments, rotation state, credit pool, and MNHF signals that consensus at that height depends on. This PR defines the evo snapshot v3 format: the versioned interchange encoding for that state, its DoS-hardened decoder, and every validation invariant that needs no chain context. It deliberately contains no chain access and no lifecycle wiring — building a snapshot from chain state and validating one against the chain come in the next PR of the series; dumptxoutset/load integration after that. Reviewing this PR is reviewing the wire format and its trust boundary, nothing else.

What was done?

  • src/evo/snapshot.{h,cpp}: the CEvoSnapshot types, canonical serialization, bounded validating deserialization, and CEvoSnapshot::Validate() (context-free invariants), plus ReconstructHistoricalMNLists(), CanonicalMNListHash(), GetEvoSnapshotHash(), and VerifyEvoSnapshotCbTx() (pure CbTx cross-checks over decoded content).
  • AssumeutxoData gains an EvoSnapshotHash field: the hard-coded expected hash of the canonical evo section, the same security anchor role hash_serialized plays for the UTXO set.
  • CDeterministicMNList gains ApplyDiffForSnapshot() and GetHeightForSnapshotCodec(); CRangesSet gains a bounded validating unserializer; OverrideStream gains GetStream() for the per-object decode budgets.
  • A ubsan suppression for vendored immer's HAMT merge (shift-base), reachable only through the unit tests' deliberately hash-colliding MN fixtures, in the style of the existing vendored-library entries.
  • A one-line explicit cast in ReadFixedBitSet whose trailing-bits mask otherwise trips clang's implicit-sign-change check for any bitset size not a multiple of eight — these tests are the first to decode such bitsets under the sanitizer job.
  • ~650 lines of unit tests: round-trip and representation independence, a golden serialized value, canonical-order and counter rejection, diff-chain determinism, cumulative decode budgets, a context-free validation matrix, claimed-size bounds, and CbTx cross-checks.

Why a bespoke codec instead of the classes' own serializers (raised in #7579 review): (1) snapshot content is hashed and cross-checked (the completion-time MN-list comparison and CbTx checks), so the encoding must be a pure function of set content — hence canonical proTxHash ordering rather than container iteration order; (2) the snapshot file is untrusted by definition and read once, so its decoder validates and bounds everything, while the EvoDB/P2P deserializers are trusted hot paths that would pay that tax per block; (3) a versioned interchange format must not silently drift when in-memory serialization changes. Note per-object serializers are reused — CDeterministicMN, commitments, and the credit pool decode through their own SERIALIZE_METHODS wrapped in a budgeted stream; only the container level (ordering, bounds, budgets) is bespoke.

Open question for reviewers (from #7579 feedback on header surface): the codec helpers are Stream-templated and therefore header-bound; I can move them into an evo::detail namespace to shrink the nominal API if preferred — say the word and it's a small mechanical commit.

How Has This Been Tested?

Full unit suite on a --enable-werror build, plus the snapshot/netinfo/util suites under a --with-sanitizers=undefined,integer build with the repo's ubsan suppressions (which is what surfaced the ReadFixedBitSet and immer items above).

Breaking Changes

None. The format is new and nothing constructs or consumes it on-chain yet; the AssumeutxoData field is populated with a null placeholder for the existing regtest entries.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation
  • I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

PastaPastaPasta and others added 2 commits August 12, 2026 22:56
…e validation

First code PR of the assumeutxo M4 series (dashpay#7579 decomposition): the versioned interchange format for Dash's evo state alongside a UTXO snapshot - canonical serialization, DoS-bounded validating decode, and every validation invariant that needs no chain context. Chain-aware building/validation and dump/load integration follow in the next PRs of the series.

Canonical ordering exists because snapshot content is hashed and cross-checked; per-object serializers are reused through a bounded stream wrapper, with bespoke code only at container level (ordering, bounds, per-entry budgets); decode-time checks deliberately stay out of the trusted hot EvoDB deserializers. AssumeutxoData gains the EvoSnapshotHash anchor the format is pinned by. Includes the aggregate rotation skip-list bound (lists accumulate across every quorum index and wrap the combined MN list), the CRangesSet bounded unserializer, and a vendored-immer shift-base ubsan suppression reachable only through the deliberately hash-colliding test fixtures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… sign change

The mask for rejecting out-of-range trailing bits promotes through operator~ to a negative int before its implicit conversion back to uint8_t, which clang's implicit-integer-sign-change check reports for every bitset whose size is not a multiple of eight. The evo snapshot unit tests are the first to deserialize such bitsets under the sanitizer job. Same bits, stated explicitly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@PastaPastaPasta
PastaPastaPasta force-pushed the feat/evo-snapshot-format branch from 3779689 to 3398d9a Compare August 13, 2026 05:13
The codec PR shipped UnserializeBounded without its unit coverage; add the malformed/canonical decode matrix and the round-trip checks from the original series.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant