feat: canonical singleton-lineage walk composed from primitive reads - #4
Conversation
`ChainSource::resolve_singleton_lineage` is the only trait method with no default body, and it is the most trust-critical one: its result IS the authority set consumers test membership against. A source backed only by primitive reads therefore had to hand-roll money-critical singleton authentication, and a second hand-rolled copy is a byte-drift bug waiting to happen. Add the composition once, behind the non-default `lineage-walk` feature: walk_singleton_lineage / walk_singleton_lineage_bounded resolve_singleton_lineage_via_walk (the one-line delegation body) LineageWalkError<E>, MAX_LINEAGE_DEPTH ChainSourceError::LineageTooDeep The walk never recognises the next coin, it derives it: each hop reads the current coin's own spend, requires the spend to be that coin's, requires the reveal to hash to that coin's puzzle hash, parses the reveal as a singleton curried to the launcher under resolution, runs the inner puzzle, and reconstructs the odd-amount successor's full puzzle hash. A coin's puzzle hash is attacker-chosen, so recognition by puzzle hash, by curried launcher id, or by selection from `coin_records_by_parent` is spoofable; admission by construction is not. Each derived successor is additionally confirmed to exist via `coin_record`, because a CLVM solution is not committed to by a coin's puzzle hash. The `CREATE_COIN` amount is decoded as SIGNED: CLVM atoms carry no sign, so the singleton melt marker -113 decodes into a u64 as 143 — an odd, positive amount indistinguishable from an ordinary recreation — and a walk that made that mistake would invent a phantom successor for every melted singleton. Bounded at MAX_LINEAGE_DEPTH spends, which refuses rather than truncating: a partial member set would make `contains` answer false for genuine members, which is a fail-open membership answer on a money path. The feature is non-default because the walk needs a CLVM evaluator; consumers that only depend on the trait gain no new transitive dependency. Closes DIG-Network/dig_ecosystem#2572 Co-Authored-By: Claude <noreply@anthropic.com>
4353778 to
0f60ca5
Compare
MichaelTaylor3d
left a comment
There was a problem hiding this comment.
CHANGES-REQUIRED - correctness gate. Head read: 0f60ca5.
The derivation core is sound. I tried to break successor selection five ways and could not: no path reads coin_records_by_parent; the successor full puzzle hash is reconstructed via SingletonArgs::curry_tree_hash(launcher_id, inner) and can never be read off an untrusted field; read_spend_of binds the reveal to the coin puzzle hash and the spend to the coin; require_coin_exists closes the fabricated-solution hole; the melt marker is decoded signed and the melt test drives a real -113 through the simulator (under u64 decoding it would read as 143, derive a coin that does not exist, and the test would fail on require_coin_exists). TooDeep is a refusal on every path and no partial SingletonLineage is ever constructed. The LineageWalkError to ChainSourceError collapse preserves the source own variant and cannot produce an absence.
The blocker is a different axis: the walk never consults CoinRecord::spent_height, while the trait itself documents coin_spend Ok(None) as "unspent OR unknown". So "I cannot read this spend" becomes "this coin is the tip" - an unknown turned into an affirmative authentication on a mint path. Details inline. Two normative SPEC 4a requirements also have no discriminating test.
Not handed to Copilot: singleton authentication on a real-money path plus test-vacuity findings go to an implementer with the threat model, not a bot.
The suite could not express the attack: the `record()` helper hardcoded `spent_height: None`, so no fixture ever had a coin that was spent while its spend was unreadable. Adds that fixture at both the launcher and mid-lineage. Refs #2572 Co-Authored-By: Claude <noreply@anthropic.com>
`ChainSource::coin_spend` returns `Ok(None)` for "unspent OR unknown", and the walk read only that, so an unreadable spend became the tip: mid-lineage it returned a stale tip (a dead singleton authenticating as live, had the lost spend been the melt), and at the launcher it degraded an unknown into "never existed" (SPEC §3). The coin's own `spent_height` separates the two; it is carried from the records the walk already reads, so no extra source call. Also: a CREATE_COIN whose arguments fail to decode is now a refusal rather than a skip, since skipping one turned an unreadable condition into a phantom melt — the same not-known-presenting-as-a-tip shape. The cycle guard is fused with the member insertion so it cannot be weakened without breaking a tested behaviour. Refs #2572 Co-Authored-By: Claude <noreply@anthropic.com>
The simulator's validator rejects a malformed condition, so an undecodable CREATE_COIN is only reachable from a lying source — these drive run_for_continuation directly. The third test is the control: well-formed melt and recreation conditions must still decode, or a decoder that refused everything would pass the other two. Refs #2572 Co-Authored-By: Claude <noreply@anthropic.com>
SPEC §4a gains requirements 6 and 7 (an unreadable spend and an undecodable CREATE_COIN are unknowns, never a tip) and states the unspent-eve return explicitly, since consumers read the contract here. MAX_LINEAGE_DEPTH is declared the ecosystem's single source of truth for the bound. Refs #2572 Co-Authored-By: Claude <noreply@anthropic.com>
…decoder guards The canonical chia melt condition carries a NIL puzzle hash — chia_sdk_types declares MeltSingleton with `puzzle_hash: () if ()` — so every melt standard chia-wallet-sdk tooling emits is `(51 () -113)`. The condition decoder forces Bytes32 before testing the melt marker, so it refuses that form: a DID or DataStore melted with standard tooling is permanently unanswerable. RED: the_canonical_nil_puzzle_hash_melt_still_decodes_as_a_melt. Co-Authored-By: Claude <noreply@anthropic.com>
The canonical chia melt condition is `(51 () -113)` — chia_sdk_types declares
`MeltSingleton { puzzle_hash: () if () }` — and it is what every melt built by
standard chia-wallet-sdk tooling emits, including dig-did's and
chip35_dl_coin's. Forcing Bytes32 as part of the CREATE_COIN argument decode
refused that form outright, so a singleton melted with standard tooling was
permanently unanswerable: Malformed forever, blaming an honest source, where
the truth is a plain final absence.
The amount is the discriminant, so it is read first; the puzzle hash is
resolved only once the amount proves the condition is an odd-amount
recreation. The refuse-on-unreadable property is preserved and now pinned from
the other side too.
Co-Authored-By: Claude <noreply@anthropic.com>
chia_protocol::Program's ToClvm uses node_from_bytes — the NON-backref reader — so a genuine singleton whose spend was serialized with back-references, the compressed form full nodes accept and block generators emit, was unresolvable and reported as Malformed: the walk blaming an honest source for chain data the chain itself considers valid. A curried singleton reveal compresses substantially (1381 -> 1289 bytes in the fixture), so this is the ordinary case, not an exotic one. Program::run reads back-references for exactly this reason; the walk now matches it. Co-Authored-By: Claude <noreply@anthropic.com>
…l-clock time Two DoS defenses the walk was missing, both of which chia-query's walk — the implementation this one is meant to replace — already carries. The allocator was hoisted outside the hop loop. clvmr's Allocator is an arena that frees nothing until dropped, so every hop's puzzle, solution and evaluation accumulated for the whole walk: 552 MB over 100,000 hops of a hostile ever-advancing chain, and the arena's own node ceiling was reached at ~74,500 hops — BEFORE MAX_LINEAGE_DEPTH — so the documented TooDeep refusal was unreachable at the default bound and a legitimate 80,000-state singleton would have been reported as malformed chain data. The allocator now lives inside successor_of, where nothing above it can hoist it back out. The hop cap bounds neither elapsed time nor per-hop CLVM cost, and ChainSource is synchronous, so a source that simply keeps answering holds the caller's thread. WalkBounds adds a wall-clock budget, defaulting to 45s to agree with chia-query, checked once per hop. A provider whose resolve_singleton_lineage is a one-line delegation inherits both guards. LineageWalkError::DeadlineExceeded is distinct from TooDeep and from Malformed, and projects to ChainSourceError::Timeout: running out of time is not evidence that the source served bad data. Both existing public signatures are unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
Two of the existing tests named one guard and were satisfied by another, which is the failure mode a coarse `matches!(_, Malformed(_))` invites when several guards produce the same variant: - a_spend_of_the_wrong_coin_fails_closed used a reveal that could not hash to the coin, so the REVEAL check fired and the coin-identity check was never reached. It now serves the genuine launcher puzzle with a workable solution, asserts the exact refusal, and carries an honest control. - the launcher puzzle-hash check was covered only by an UNSPENT ordinary coin, whose Ok(None) arrives through the at-launcher branch with or without the check. A SPENT non-launcher whose spend would otherwise mint a good eve is the only fixture that distinguishes them. Two guards had no test at all: the curried-launcher-id filter (reached only by a well-formed singleton of a DIFFERENT launcher — every other non-singleton fixture is refused one step earlier at the parse) and the more-than-one-odd-child refusal. Each of the four now goes red when, and only when, its own guard is deleted. Co-Authored-By: Claude <noreply@anthropic.com>
…oth walk bounds SPEC §4a gains four normative requirements the walk now satisfies: decode the CREATE_COIN amount before the puzzle hash (so the canonical `(51 () -113)` melt decodes), deserialize programs with the back-reference reader, refuse past a wall-clock budget as well as a hop cap, and start each hop with a fresh CLVM allocator. Other providers are told in §7 to delegate to this walk, so a requirement absent from the SPEC is a requirement a reimplementation will miss. Co-Authored-By: Claude <noreply@anthropic.com>
…ompression bomb `program_tree_hash` hashed a puzzle reveal with `clvm_utils::tree_hash`, a plain explicit-stack traversal with no memoization. Since `6b2c2b2` the reveal is deserialized with `node_from_bytes_backrefs`, which decodes back-references into a shared DAG — so a `k`-level self-referential DAG costs `2^k` hash operations. That makes a tiny puzzle reveal a decompression bomb, and it detonates on the FIRST hop at zero cost to the attacker: a source that serves an honest launcher record can answer `coin_spend` with the bomb, and `read_spend_of` must hash the reveal before it can compare it to the coin's puzzle hash. The walk's wall-clock budget does not help, because it is checked between hops rather than inside one. Measured at depth 24 (73 serialized bytes), debug: 78.2 s before, 0.15 s after. `tree_hash_from_bytes` is back-reference-aware AND memoizing — the identical call `chia-peer` makes at `provider.rs:215` — so compressed-reveal support, the point of `6b2c2b2`, is preserved. The new control test asserts that explicitly, so the bomb cannot be "fixed" by banning back-references again. Co-Authored-By: Claude <noreply@anthropic.com>
…e memoizing hash left open The previous fix memoized the reveal-binding hash. That closed one detonation site and left a second, worse one: chia_sdk_driver::Puzzle::parse calls the NON-memoizing clvm_utils::tree_hash, and no cache protects it at any depth. Currying the same back-reference bomb as the INNER puzzle of an otherwise genuine singleton walks straight past the binding check -- the reveal really does hash to the eve's puzzle hash, because the walk DERIVES that hash from the bomb's own tree hash -- and detonates one hop later. Measured through the public API, release profile: a 1,120-byte reveal costs 75 seconds, doubling every three bytes on the wire. A cap on the reveal's serialized length cannot see that attack; every one of those reveals is about a kilobyte. So the bound is on the EXPANSION instead, computed in time linear in the DAG by a memoized saturating traversal, and placed ahead of every use of the bytes. Co-Authored-By: Claude <noreply@anthropic.com>
…efore the parity skip Three hardening changes folded into the reveal-bound round. run_for_continuation evaluated at MAINNET_CONSTANTS.max_block_cost_clvm -- a whole block -- against a solution bound to nothing, so one hop could burn an entire block's evaluation and the between-hops wall-clock check was a promise the walk could not keep. It now evaluates at an explicit MAX_HOP_CLVM_COST, chosen from measured honest hops (launcher spend 11,932; p2 inner spend 18,092). The Bytes32 decode moves above the even-amount skip. Melts still return earlier, so the melt fix is preserved, while an unreadable puzzle hash is refused for every non-melt CREATE_COIN rather than only the odd-amount ones. WalkBounds gains #[non_exhaustive], private fields and a clamp on max_hops, so no struct literal can silently disable either guard. It has no published consumers: walk.rs does not exist on main and 0.3.0 is the newest release. SPEC.md sections 5, 5a and 5b now state the real guarantee -- budget plus one worst-case hop, with the wall-clock check named as the backstop it is. Co-Authored-By: Claude <noreply@anthropic.com>
Consider publishing this as 0.3.1, not 0.4.0 — the change appears fully semver-additiveRaising this from the dig-app side, because the version choice here decides whether adopting this walk is one change or four coordinated releases. Not a review blocker — filing as a plain comment deliberately so it cannot hold up the merge. Why 0.4.0 is expensiveThree crates currently pin
For a Adopting 0.4.0 therefore requires a release-first cascade: csi 0.4.0 → dig-did 0.7.0 → dig-account 0.12.0 → dig-app. Four releases, three of them breaking, to adopt a change that breaks nothing. Why it looks additive
What 0.3.1 would buydig-app adopts by adding Two caveats worth checking before changing the number
If there is a deliberate breaking change I have missed, disregard this — I checked the public surface mechanically rather than by reading intent, and the author's judgement wins. The Context: this walk is the sole remaining blocker on a real mainnet DID+profile mint from the dig-app UI (dig_ecosystem#2398 / #2572). |
Independent adversarial gate: PASS — and both unification caveats on the 0.3.1 question are now closedFollowing up my earlier comment with the result of a fresh independent gate (separate context, prompted to refute) plus one fact it could not check that I have since verified. The gate could not break itWhole suite built and run in release with Highlights, all verified mechanism-first rather than from doc comments:
On the version: the gate independently reached 0.3.1 as wellIts strongest argument is better than mine — closed by file absence, not inspection:
So the trait, every default-method body, and every other public type are byte-identical. Adding to that: Both unification caveats are now closedThe gate flagged two risks it could not verify. I checked both against published crates.io manifests rather than the working checkouts, which are stale (the shared dig-account checkout still shows 0.5.0 on chia 0.26 — worth knowing before anyone reads a dep line from disk).
No duplicate chia tree. And because dig-account 0.11.3 already depends on NetNothing here blocks the merge. The only suggestion is the version number, and since 0.4.0 is unpublished, changing it is free — whereas shipping it costs a four-release cascade (csi → dig-did 0.7 → dig-account 0.12 → dig-app) to adopt a change that breaks nothing. Three non-blocking hardening notes have been filed as tickets rather than raised here: the solution is not expansion-bounded (latent — no bomb is currently reachable, but a future change that tree-hashes a solution reopens round 3 with nothing red), |
…bump The `lineage-walk` addition changes no existing item: `src/source.rs`, `src/lineage.rs`, `src/provider.rs` and `src/record.rs` are untouched, so the `ChainSource` trait, every default-method body and every public type are byte-identical. All new code is behind the non-default `lineage-walk` feature, all seven new dependencies are optional, `rust-version` and the default feature set are unchanged, and the two new `ChainSourceError` variants land on an enum that was already `#[non_exhaustive]` at 0.3.0. 0.4.0 would break the `"0.3"` requirement held by dig-app, dig-account 0.11.3 and dig-did 0.6.0, splitting `ChainSource` into two incompatible traits and forcing a four-release cascade to adopt a change that breaks nothing. Refs: dig_ecosystem#2572, dig_ecosystem#2398 Co-Authored-By: Claude <noreply@anthropic.com>
The canonical singleton lineage walk (feature
lineage-walk)Adds
walk_singleton_lineageand friends: the ONE launcher → tip singleton walk, composed from theChainSourceprimitives, so a provider'sresolve_singleton_lineageis a one-line delegationinstead of a hand-rolled copy of money-critical authentication.
Version stays 0.4.0 — not yet published.
Round 3: the previous fix was a mitigation, and the attack still worked
The prior round memoized the reveal-binding hash. That closed one detonation site and left a second,
worse one, which this round closes.
The bypass, measured end-to-end through the public API (release profile)
chia_sdk_driver::Puzzle::parsecalls the non-memoizingclvm_utils::tree_hash. Curry the sameback-reference bomb as the inner puzzle of an otherwise genuine singleton and the binding hash
sails through — the reveal really does hash to the eve's puzzle hash, because the walk derived
that puzzle hash from the bomb's own tree hash — and the spend then reaches a hash no cache
protects, at any depth:
Roughly three bytes on the wire per doubling, unbounded. Reachability is one hop deeper than the
original: the same lying source answers the
coin_recordthat binds the eve to "real" chain state,so it still costs the attacker no funded coin and no owned singleton.
This is why a
puzzle_revealbyte-length cap — the obvious fix — was not implemented. Every oneof those reveals is about a kilobyte. A length cap sized for an honest singleton cannot see this
attack at all.
The fix: bound the EXPANSION, ahead of every use of the bytes
MAX_REVEAL_EXPANDED_BYTES = 4 MiB, checked inread_spend_ofbefore the reveal is hashed,parsed or run. The expansion is computed by a memoized saturating traversal in time linear in the
DAG, so a bomb saturates instead of being counted. Amplification is capped at 1 by construction: no
serialized size, at any depth, buys more work than the bound.
Honest reveal sizes measured first, and the bound chosen from them (curried from the canonical
chia puzzles; expansion is the SHA-256 input the tree hash consumes):
p2_delegated_puzzle_or_hidden_puzzlealonesingleton(p2)singleton(did_innerpuz(p2)), empty metadatasingleton(nft state + ownership + royalty), 8 long URIssingleton(did_innerpuz(p2))with 8 KB curried metadata4 MiB is 30x the heaviest realistic reveal and 6.6x a deliberately absurd one, because the
nearest wrong fix is a bound so tight it refuses an honest singleton on a money path. It still caps a
hop's hashing at roughly two milliseconds, leaving the wall-clock budget the effective outer bound.
The refusal is distinguishable, not
MalformedNew
LineageWalkError::RevealTooLarge { coin_id, limit }, projecting toChainSourceError::RevealTooLarge { limit }. The chain data may be perfectly well-formed and merelytoo large to authenticate; calling that corruption would be a lie, and would hide the one thing a
consumer can act on.
Before / after at the auditor's depth 65,560, on this machine
~1,490x, and the residual is now flat in depth rather than doubling every 3 bytes.
Also in this round
run_puzzleevaluated atmax_block_cost_clvm(11,000,000,000) against a solution bound to nothing — only the reveal ishash-bound to the coin — so one hop could burn a whole block. Now evaluated at an explicit
MAX_HOP_CLVM_COST = 100,000,000, chosen from measured honest hops (launcher spend 11,932, p2inner spend 18,092; ~5,500x headroom, ~110x below a block).
walk.rsandSPEC.md§5 nowstate the real bound:
budget + one worst-case hop, with the wall-clock check named as thebackstop it is — a guarantee that is vacuous unless one hop's cost is itself bounded, which §5a
and §5b now require.
Bytes32decode sat afterthe even-amount parity skip, so an even-amount
CREATE_COIN's puzzle hash went unvalidated. Movedabove the skip (melts still return earlier, so the melt fix is preserved).
WalkBoundsfootgun closed:#[non_exhaustive], private fields with accessors, andhops()clamps toMAX_LINEAGE_DEPTH.WalkBounds { max_hops: usize::MAX, budget: Duration::MAX }disabled both guards through a struct literal with nothing in the diff to signal it. Safe to do
now:
walk.rsdoes not exist onmainand 0.3.0 is the newest published version, so it has zeropublished consumers.
successor_of's per-hop allocator is now mechanically pinned. Its doc claimed the reset"cannot be quietly undone by moving a line" — true of a line, but a hoist is a 3-line change of
exactly the shape a "stop reallocating per hop" optimisation takes, and nothing went red. A test
asserts the signature declares no
Allocatorparameter; the doc is softened to match what isactually enforced.
Not done, deliberately: replacing the hand-rolled melt discriminator with
chia_sdk_types::Condition.Condition::MeltSingletonmatches onlypuzzle_hash: () if (), so amelt carrying a 32-byte hash would fall through to
CreateCoin, whoseamountisu64, turning-113into143and reinstating the phantom-successor CRITICAL fixed in an earlier round. Bothmelt forms must decode.
Verification
require_expandable_revealcall fails bothbomb tests; the inner-bomb test burns 278 s before failing. Restored, suite green.
unfixed cost there is 1.91 s against a 5 s bound, so the test passed with the defect present.
At depth 30 the unfixed costs are ~120 s (ladder) and ~300 s (inner bomb) — ~24x and ~60x margins,
in release. The doc comment's "three orders of magnitude" claim was wrong and is replaced with the
measured figures and the profile they were taken in.
be admitted, one byte over must be refused. Plus a saturation test (2^40 must not be computed) and
an honest-size control that must keep passing.
satisfies the ladder test and leaves the inner-bomb test detonating, so the second fixture is what
proves where the bound sits, not merely that it exists.
a_genuinely_backref_compressed_reveal_is_still_acceptedstill passes — the bound must notre-ban back-references.
cargo clippy --all-targets -- -D warningsclean. All 8 required checks green.Blast radius
gitnexus was unavailable —
npx gitnexus analyzesegfaults in this worktree — so the radius wasestablished with ripgrep plus direct reads, per the fallback that §2.0 permits. Stated rather than
skipped.
Symbols changed:
read_spend_of,run_for_continuation,program_tree_hash(doc),WalkBounds,LineageWalkError,ChainSourceError, plus the newexpanded_hash_input_bytes,require_expandable_revealand two constants. Every caller is insidesrc/walk.rs(102 references),src/lib.rs(re-export) and two test files; nothing else in the repo touches them.Risk: MEDIUM, no HIGH/CRITICAL. Five crates depend on this one, but the
lineage-walksurface isnew in this PR and has no published consumers. The only change to an already-published API is the
additive
ChainSourceError::RevealTooLargevariant, which is safe because that enum is#[non_exhaustive]— consumers were already required to carry a wildcard arm.