refactor(forks): nest forks/ under new spec/ subpackage#788
Merged
tcoratger merged 9 commits intoMay 28, 2026
Conversation
Group per-fork consensus rules under a dedicated lean_spec.spec namespace so the protocol specification has a clear home as more spec content (SSZ, chain, validator, etc.) potentially moves alongside it. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Move the two cryptographic primitive subspecs into the spec/ subpackage
created in the previous commit:
- subspecs/koalabear/field.py -> spec/crypto/koalabear.py
- subspecs/poseidon1/{constants,permutation}.py -> spec/crypto/poseidon.py
Each primitive collapses into a single module: koalabear loses its
re-exporting __init__, and poseidon merges its round-constants table
with the permutation engine. The Poseidon1 class is renamed to Poseidon
(and Poseidon1Params -> PoseidonParams) since the file no longer carries
the variant number; PARAMS_16/PARAMS_24 keep their names.
All import sites in src/, tests/, and packages/testing/ are updated.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…on.py Merge the four-file SSZ hashing subspec into a single module that lives alongside the other primitives under spec/crypto/. The hash-tree-root dispatch, the binary-tree merkleizer, the length-mix helper, and the two chunk-width constants now all sit in one place; the cross-file imports between hash.py, merkleization.py, and constants.py disappear. All import sites in src/, tests/, and packages/testing/ are updated to import from spec.crypto.merkleization. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the three-piece machinery (named max-depth constant, builder
function, module-level call) with one itertools.accumulate that folds
the recurrence h_{d+1} = sha256(h_d || h_d) over the all-zero seed.
The cache now covers depth 64 unconditionally, so _zero_tree_root drops
the past-cache fallback loop and becomes a direct table lookup. The two
tests that exercised the fallback are deleted with the dead code.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ments Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two related reorgs that finish carving out spec/ vs runtime/:
- XMSS is a cryptographic primitive specification, structurally identical
to the koalabear/poseidon/merkleization siblings, so move the entire
subpackage to lean_spec.spec.crypto.xmss alongside them.
- Everything left under subspecs/ is reference-node runtime (api,
networking, storage, sync, validator, observability, metrics, plus the
chain/genesis fuzzy middle ground and the Node orchestrator itself).
Rename it to lean_spec.node so the package name reflects what it is.
The old inner subspecs/node/ subpackage is dissolved in the same move:
its node.py and anchor.py lift up by one level so the new layout exposes
lean_spec.node.{Node, NodeConfig, anchor.*} directly instead of nesting
them inside lean_spec.node.node.*.
All import sites in src/, tests/, and packages/testing/ are updated, the
per-file ruff ignore in pyproject.toml moves with xmss, and the xmss
internal relative imports are repointed to the new neighbors.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Snappy framing is a wire-format runtime concern used by gossip and reqresp, not a piece of the protocol specification. Group it with the other runtime services under lean_spec.node. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ean_spec.base The 8 SSZ-primitive modules under lean_spec.types (bitfields, boolean, byte_arrays, collections, container, exceptions, ssz_base, uint) move to lean_spec.spec.ssz alongside the other protocol-spec subpackages. What stays under lean_spec.types is the domain layer: Slot, Checkpoint, ValidatorIndex, SubnetId, AggregationBits, the RLP helpers, and the participation bits — types built on top of SSZ rather than defining it. base.py (CamelModel, StrictBaseModel) is not SSZ-specific and is used by both SSZ models and non-SSZ Pydantic models. Moving it would create a spec/ssz/ssz_base.py → types.base → types/__init__.py → spec/ssz cycle, so it lifts one level up to lean_spec.base, sitting next to config.py and log.py as generic Pydantic infrastructure. All 135 SSZ-touching import sites are split where they mix SSZ and domain symbols, and 21 sites importing StrictBaseModel/CamelModel are repointed at lean_spec.base. No backward-compat re-exports. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The unit test tree was lagging behind the recent source reorgs. Bring
tests/lean_spec/ into 1:1 correspondence with src/lean_spec/:
- tests/lean_spec/types/{test_bitfields,test_boolean,test_byte_arrays,
test_collections,test_container,test_ssz_base,test_uint}.py
-> tests/lean_spec/spec/ssz/
- tests/lean_spec/types/test_base.py -> tests/lean_spec/test_base.py
- tests/lean_spec/forks/ -> tests/lean_spec/spec/forks/
- tests/lean_spec/subspecs/{api,chain,genesis,metrics,networking,
observability,storage,sync,validator}/ -> tests/lean_spec/node/
- tests/lean_spec/subspecs/node/{test_anchor,test_node}.py dissolved up
into tests/lean_spec/node/
- tests/lean_spec/subspecs/koalabear/test_field.py
-> tests/lean_spec/spec/crypto/test_koalabear.py
- tests/lean_spec/subspecs/poseidon1/test_permutation.py
-> tests/lean_spec/spec/crypto/test_poseidon.py
- tests/lean_spec/subspecs/ssz/{test_hash,test_merkleization}.py
-> tests/lean_spec/spec/crypto/
- tests/lean_spec/subspecs/xmss/ -> tests/lean_spec/spec/crypto/xmss/
- tests/lean_spec/subspecs/containers/test_attestation_aggregation.py
-> tests/lean_spec/spec/forks/lstar/
- tests/lean_spec/snappy/ -> tests/lean_spec/node/snappy/
- tests/consensus/{lstar,devnet}/poseidon1 -> .../poseidon
- tests/lean_spec/subspecs/conftest.py -> tests/lean_spec/node/conftest.py
Justfile paths updated: codespell --skip for the snappy testdata
fixture, and the test-consensus recipe now points at the new layout.
Side fix: the node/__init__.py eager re-export of Node/NodeConfig was
turning any import of lean_spec.node.<anything> into a load of the
whole api -> networking -> reqresp.handler chain, which then tries to
import SignedBlock from lean_spec.spec.forks while spec.forks is
mid-init. The three call sites switch to the explicit module path
lean_spec.node.node, and the __init__.py drops the re-export.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
tcoratger
added a commit
to unnawut/leanSpec
that referenced
this pull request
May 30, 2026
…renames Rebase onto main and apply the renames/cleanups landed since the branch was opened: - PR leanEthereum#799: TypeOneMultiSignature → SingleMessageAggregate, proof_type literal "type_1" → "single_message", file renames test_type_1_{valid,invalid}.py → test_single_message_{valid,invalid}.py. - PR leanEthereum#800: validator_ids → validator_indices, with_validator_id → with_validator_index, vid/pubkey expansions. - post-leanEthereum#788/leanEthereum#790/leanEthereum#796 imports: lean_spec.spec.forks / .spec.ssz / .spec.crypto.*. Replace the stringly-typed tamper dict with a Pydantic discriminated union (RebindToAlternateHeadRoot, IncrementEmittedSlot, SwapParticipantPublicKey). The match dispatch on the typed variants drops the two type: ignore[index] casts and lets the test sites read tamper=SwapParticipantPublicKey(index=0, with_validator_index=1) instead of a magic-string dict. Inline the four single-call helpers (_apply_tamper plus three _tamper_*) and the verification helper into make_fixture so the four phases — generate / tamper / verify / publish — are visible in one method. Drop both model_copy(update=...) calls per leanEthereum#789: direct field construction for the frozen AttestationData rebuild, direct field assignment for the mutable fixture self-update. Replace the internal dict[str, Any] bundle with named locals; the bundle never escapes make_fixture, so the dict adds nothing. Guard SwapParticipantPublicKey against silent no-op swaps where the replacement key happens to equal the original. Broaden the verifier exception catch to surface unexpected exception types as "expected X got Y" instead of crashing the filler. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
tcoratger
added a commit
that referenced
this pull request
May 30, 2026
…heck proof verification (#786) * test(consensus): add VerifyProofsTest fixture and Type-1 valid vectors Introduces a new consensus fixture format that emits self-contained multi-signature verification vectors so cross-client implementations can run their own Type-1 verifier and compare outcomes. Three positive vectors land alongside the fixture: a single-validator baseline, a four-validator all-participating case, and a four-validator non-contiguous bitfield case ([1, 0, 1, 1]). The fixture also surfaces the spec-layer binding between attestation data and proof: clients recompute hash_tree_root(attestation_data) and must match the emitted message field before running the verifier. * test(consensus): add Type-1 verify_proofs rejection vectors Adds four negative vectors exercising spec-layer bindings between inputs and the multi-signature proof. Each vector uses a tamper operation on the fixture to produce a structurally valid bundle that must be rejected by a conformant verifier: - wrong_message: proof bound to an alternate head root inside the attestation data - wrong_slot: emitted slot field shifted while the proof binding stays on the original slot - wrong_public_keys: one emitted pubkey replaced with another validator's - aggregation_bits_length_mismatch: emitted bits truncated while the pubkey count stays unchanged Vectors covering malformed or truncated proof bytes are intentionally out of scope: leanSpec consumes the multi-signature primitive as a black box and primitive integrity belongs to its own conformance suite. Pubkey ordering is also not a binding to test: the aggregator sorts participants internally, so the verifier is order-insensitive. * test(consensus): align VerifyProofsTest with sibling fixture conventions Brings the new fixture in line with the patterns the other consensus test fixtures follow: - Drop ``from __future__ import annotations`` (PR #759 removed it from Pydantic-defining files); quote the one self-reference instead. - Replace the bespoke ``expect_valid: bool`` field with the inherited ``expect_exception`` field already used by SSZTest, NetworkingCodec, and VerifySignaturesTest. Rejection vectors now pin ``AggregationError`` and the framework serializes the class name to JSON. - Switch the tamper dispatch in ``_apply_tamper`` from ``if/elif`` to ``match/case`` to follow the pattern in slot_clock and networking_codec. - Expand the module-level docstring from one line to a short paragraph describing what the fixture covers. - Normalize the ``public_keys`` default from ``[]`` to ``| None = None`` to match every other output field on the model. * test(consensus): drop aggregation_bits_length_mismatch rejection vector The check that fires here is the early-reject in the spec wrapper's verify method (len(public_keys) != participants.count(True)), not a consensus-critical binding. In real consensus the inconsistency cannot arise because clients resolve public keys from the bitfield plus the validator registry as one operation. A client that did pass a wrong pubkey count would also be rejected by the underlying recursive verifier on its internal pubkey-set commitment, so the wrapper check is at best an early exit with a nicer error message. The remaining three rejection vectors still exercise the meaningful spec-layer bindings: message hash, slot, and pubkey set. * refactor(consensus): tighten VerifyProofsTest and absorb post-rebase renames Rebase onto main and apply the renames/cleanups landed since the branch was opened: - PR #799: TypeOneMultiSignature → SingleMessageAggregate, proof_type literal "type_1" → "single_message", file renames test_type_1_{valid,invalid}.py → test_single_message_{valid,invalid}.py. - PR #800: validator_ids → validator_indices, with_validator_id → with_validator_index, vid/pubkey expansions. - post-#788/#790/#796 imports: lean_spec.spec.forks / .spec.ssz / .spec.crypto.*. Replace the stringly-typed tamper dict with a Pydantic discriminated union (RebindToAlternateHeadRoot, IncrementEmittedSlot, SwapParticipantPublicKey). The match dispatch on the typed variants drops the two type: ignore[index] casts and lets the test sites read tamper=SwapParticipantPublicKey(index=0, with_validator_index=1) instead of a magic-string dict. Inline the four single-call helpers (_apply_tamper plus three _tamper_*) and the verification helper into make_fixture so the four phases — generate / tamper / verify / publish — are visible in one method. Drop both model_copy(update=...) calls per #789: direct field construction for the frozen AttestationData rebuild, direct field assignment for the mutable fixture self-update. Replace the internal dict[str, Any] bundle with named locals; the bundle never escapes make_fixture, so the dict adds nothing. Guard SwapParticipantPublicKey against silent no-op swaps where the replacement key happens to equal the original. Broaden the verifier exception catch to surface unexpected exception types as "expected X got Y" instead of crashing the filler. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
src/lean_spec/forks/tosrc/lean_spec/spec/forks/so per-fork consensus rules live under a dedicatedspec/subpackage.lean_spec.forks→lean_spec.spec.forksacrosssrc/,tests/, andpackages/testing/(and the_FORBIDDEN_FORK_PREFIXESstring literal in the fork-protocol guard test).src/lean_spec/spec/__init__.py.paths:glob and the architecture diagram in.claude/rules/ssz-patterns.mdto the new location.Motivation
The top-level
lean_specpackage mixes the protocol specification (forks/,types/, parts ofsubspecs/) with runtime concerns (cli/,snappy/, parts ofsubspecs/). Carving out a dedicatedspec/namespace gives the protocol specification a clear home and a place to grow as more spec content (SSZ, chain, validator, …) potentially moves alongside it.This first step intentionally only moves
forks/.types/is foundational (imported everywhere, including fromsubspecs/ssz/which is itself spec), so leaving it at the top level avoids creating an inverted dependency on day one. A broader spec-vs-runtime split can be revisited later.Test plan
just check— ruff lint, ruff format, ty, codespell, mdformatuv sync --reinstall-package lean-specsucceeds against the new layout🤖 Generated with Claude Code