Skip to content

Aggregated block proof - devnet5 #717

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
anshalshukla wants to merge 22 commits into
base: main
Choose a base branch
from
Open

Aggregated block proof - devnet5 #717

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
0ab09dd
dummy type 1 and type 2 aggregation with block proofs
anshalshukla May 11, 2026
e0006b2
integrate bindings, fix tests
anshalshukla May 14, 2026
6fd8f6d
merge main
anshalshukla May 14, 2026
c7c93d5
update bindings
anshalshukla May 14, 2026
6dc20ba
update bindings
anshalshukla May 14, 2026
4d0630b
cleanup
anshalshukla May 14, 2026
e31c1b2
merge main
anshalshukla May 15, 2026
570abaf
fix lint
anshalshukla May 15, 2026
4e3ef42
limit max attestation data to 8
anshalshukla May 15, 2026
ccdb16d
update CI params, remove parallelization in fill
anshalshukla May 15, 2026
746f2ae
fix reaggregation logic, cleanup
anshalshukla May 15, 2026
401a145
merge main
anshalshukla May 16, 2026
db235dc
reduce max proof size to 500KiB
anshalshukla May 16, 2026
ed0e53c
switch few CI jobs to macos
anshalshukla May 16, 2026
e82be6b
merge main
anshalshukla May 18, 2026
384758d
fix failing tc post main merge, minor refactor
anshalshukla May 18, 2026
8f9bdc1
deconstruct even in case of proposer, not just for aggregators
anshalshukla May 18, 2026
7fa0eba
address comments
anshalshukla May 18, 2026
47a06b2
block deconstruction
anshalshukla May 18, 2026
0e180f5
Merge branch 'main' into aggregated-block-proof
tcoratger May 19, 2026
369e40a
address review feedback
tcoratger May 19, 2026
1db9e4d
address second-pass review
tcoratger May 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,9 +76,15 @@ jobs:
- name: Run tests
run: just test

# coverage-gate, fill-tests and interop-tests run on macOS because the
# lean_multisig_py prover's process-global setup is corrupted by xdist
# parallelism; these jobs must run serially and macOS runners were the
# stable choice during bring-up. This is an upstream limitation, not a
# preference: revisit (and prefer ubuntu for cost) once the prover setup
# is per-process safe. See the matching note in the Justfile fill-ci.
coverage-gate:
name: Coverage gate - Python 3.12
runs-on: ubuntu-latest
runs-on: macos-latest
Comment thread
tcoratger marked this conversation as resolved.
steps:
- name: Checkout leanSpec
uses: actions/checkout@v4
Expand All @@ -104,7 +110,7 @@ jobs:

fill-tests:
name: Fill test fixtures - Python 3.14
runs-on: ubuntu-latest
runs-on: macos-latest
steps:
- name: Checkout leanSpec
uses: actions/checkout@v4
Expand All @@ -130,7 +136,7 @@ jobs:

interop-tests:
name: Interop tests - Multi-node consensus
runs-on: ubuntu-latest
runs-on: macos-latest
timeout-minutes: 10
steps:
- name: Checkout leanSpec
Expand Down
6 changes: 5 additions & 1 deletion Justfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,9 +81,13 @@ test-consensus *args:
uv run --group test pytest -n auto --maxprocesses=10 --durations=10 --dist=worksteal tests/lean_spec/subspecs/containers tests/lean_spec/subspecs/forkchoice tests/lean_spec/subspecs/networking "$@"

# Canonical CI fixture run; contributors should use `uv run fill` directly
# Runs serially (no -n auto): xdist workers race on the lean_multisig_py
# prover's process-global setup, which corrupts proofs intermittently.
# This is an upstream limitation; restore -n auto once the prover setup
# is per-process safe. See the matching note in .github/workflows/ci.yml.
[group('tests'), private]
fill-ci *args:
uv run --group test fill --fork=Lstar --clean -n auto "$@"
uv run --group test fill --fork=Lstar --clean "$@"
Comment thread
tcoratger marked this conversation as resolved.

# Run API conformance tests against an external client
[group('tests')]
Expand Down
71 changes: 38 additions & 33 deletions packages/testing/src/consensus_testing/keys.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,10 @@

from lean_spec.config import LEAN_ENV
from lean_spec.forks.lstar.containers import AttestationData
from lean_spec.forks.lstar.containers.block.types import (
AggregatedAttestations,
AttestationSignatures,
)
from lean_spec.forks.lstar.containers.block.types import AggregatedAttestations
from lean_spec.subspecs.koalabear import Fp
from lean_spec.subspecs.ssz.hash import hash_tree_root
from lean_spec.subspecs.xmss.aggregation import AggregatedSignatureProof
from lean_spec.subspecs.xmss.aggregation import TypeOneMultiSignature
from lean_spec.subspecs.xmss.constants import TARGET_CONFIG
from lean_spec.subspecs.xmss.containers import (
PublicKey,
Expand All @@ -67,7 +64,13 @@
HashTreeOpening,
Randomness,
)
from lean_spec.types import Bytes32, Slot, Uint64, ValidatorIndex, ValidatorIndices
from lean_spec.types import (
Bytes32,
Slot,
Uint64,
ValidatorIndex,
ValidatorIndices,
)

KeyRole = Literal["attestation", "proposal"]
"""Discriminator for which signing role's key to load from a validator key pair."""
Expand Down Expand Up @@ -514,18 +517,21 @@ def sign_and_aggregate(
self,
validator_ids: list[ValidatorIndex],
attestation_data: AttestationData,
) -> AggregatedSignatureProof:
) -> TypeOneMultiSignature:
"""
Sign attestation data with each validator and aggregate into a single proof.
Sign attestation data with each validator and aggregate into a Type-1 proof.
Convenience method for the common sign-each-validator-then-aggregate pattern.
Each validator's XMSS attestation key signs the attestation data
root. The signatures are then handed to the multi-signature
binding to produce a single cryptographically valid Type-1 proof
binding all participants to (data, slot).
Args:
validator_ids: Validators to sign with.
attestation_data: The attestation data to sign.
Returns:
Aggregated signature proof combining all validators' signatures.
Cryptographically valid Type-1 proof covering validator_ids.
"""
raw_xmss = [
(
Expand All @@ -534,46 +540,44 @@ def sign_and_aggregate(
)
for vid in validator_ids
]

xmss_participants = ValidatorIndices(data=validator_ids).to_aggregation_bits()

return AggregatedSignatureProof.aggregate(
xmss_participants=xmss_participants,
return TypeOneMultiSignature.aggregate(
xmss_participants=ValidatorIndices(data=validator_ids).to_aggregation_bits(),
children=[],
raw_xmss=raw_xmss,
message=hash_tree_root(attestation_data),
slot=attestation_data.slot,
)

def build_attestation_signatures(
def build_attestation_proofs(
self,
aggregated_attestations: AggregatedAttestations,
signature_lookup: Mapping[AttestationData, Mapping[ValidatorIndex, Signature]]
| None = None,
) -> AttestationSignatures:
) -> list[TypeOneMultiSignature]:
"""
Produce aggregated signature proofs for a list of attestations.
Produce Type-1 proofs aligned with the given attestations.
For each aggregated attestation:
1. Identify participating validators from the aggregation bitfield
2. Collect each participant's public key and individual signature
3. Combine them into a single aggregated proof for the leanVM verifier
1. Identify participating validators from the aggregation bitfield.
2. Collect each participant's attestation public key and signature.
3. Combine them into a single Type-1 single-message proof via the
multi-signature binding.
Pre-computed signatures can be supplied via the lookup to avoid
redundant signing. Missing signatures are computed on the fly.
redundant signing. Missing entries are signed on the fly.
Args:
aggregated_attestations: Attestations with aggregation bitfields set.
signature_lookup: Optional pre-computed signatures keyed by
attestation data then validator index.
Returns:
One aggregated signature proof per attestation.
One Type-1 single-message proof per attestation, parallel to the input.
"""
lookup = signature_lookup or {}

proofs: list[AggregatedSignatureProof] = []
proofs: list[TypeOneMultiSignature] = []
for agg in aggregated_attestations:
# Decode which validators participated from the bitfield.
validator_ids = agg.aggregation_bits.to_validator_indices()
Expand All @@ -582,7 +586,7 @@ def build_attestation_signatures(
# Fall back to signing on the fly for any missing entries.
sigs_for_data = lookup.get(agg.data, {})

# Collect the attestation public key for each participant.
# Collect the attestation public keys for each participant.
public_keys = [self.get_public_keys(vid)[0] for vid in validator_ids]

# Gather individual signatures, computing any that are missing.
Expand All @@ -593,16 +597,17 @@ def build_attestation_signatures(

# Produce a single aggregated proof that the leanVM can verify
# in one pass over all participants.
proof = AggregatedSignatureProof.aggregate(
xmss_participants=agg.aggregation_bits,
children=[],
raw_xmss=list(zip(public_keys, signatures, strict=True)),
message=hash_tree_root(agg.data),
slot=agg.data.slot,
proofs.append(
TypeOneMultiSignature.aggregate(
children=[],
raw_xmss=list(zip(public_keys, signatures, strict=True)),
xmss_participants=agg.aggregation_bits,
message=hash_tree_root(agg.data),
slot=agg.data.slot,
)
)
proofs.append(proof)

return AttestationSignatures(data=proofs)
return proofs


def _generate_single_keypair(
Expand Down
2 changes: 1 addition & 1 deletion packages/testing/src/consensus_testing/test_fixtures/fork_choice.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -300,7 +300,7 @@ def make_fixture(self) -> Self:
case BlockStep():
# Build a complete signed block from the lightweight spec.
# The spec contains minimal fields; we fill the rest.
signed_block = step.block.build_signed_block_with_store(
signed_block, store = step.block.build_signed_block_with_store(
store, self._block_registry, key_manager, self.lean_env
)

Expand Down
8 changes: 4 additions & 4 deletions packages/testing/src/consensus_testing/test_fixtures/state_transition.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from lean_spec.forks.lstar.containers.state import State
from lean_spec.forks.lstar.spec import LstarSpec
from lean_spec.subspecs.ssz.hash import hash_tree_root
from lean_spec.subspecs.xmss.aggregation import AggregatedSignatureProof
from lean_spec.subspecs.xmss.aggregation import TypeOneMultiSignature
from lean_spec.types import Bytes32, ValidatorIndices

from ..keys import XmssKeyManager
Expand Down Expand Up @@ -250,7 +250,7 @@ def _build_block_from_spec(

# Path 3: normal block construction via the spec's builder.
else:
aggregated_payloads: dict[AttestationData, set[AggregatedSignatureProof]] = {}
aggregated_payloads: dict[AttestationData, set[TypeOneMultiSignature]] = {}
if spec.attestations:
aggregated_payloads = StateTransitionTest._build_aggregated_payloads_from_spec(
spec.attestations, state, block_registry
Expand Down Expand Up @@ -304,7 +304,7 @@ def _build_aggregated_payloads_from_spec(
attestation_specs: list[AggregatedAttestationSpec],
state: State,
block_registry: dict[str, Block],
) -> dict[AttestationData, set[AggregatedSignatureProof]]:
) -> dict[AttestationData, set[TypeOneMultiSignature]]:
"""
Build aggregated signature payloads from attestation specifications.

Expand All @@ -320,7 +320,7 @@ def _build_aggregated_payloads_from_spec(
# XMSS keys require precomputation up to the highest slot used.
max_slot = max(spec.slot for spec in attestation_specs)
key_manager = XmssKeyManager.shared(max_slot=max_slot)
payloads: dict[AttestationData, set[AggregatedSignatureProof]] = {}
payloads: dict[AttestationData, set[TypeOneMultiSignature]] = {}

for spec in attestation_specs:
if not spec.valid_signature:
Expand Down
83 changes: 60 additions & 23 deletions packages/testing/src/consensus_testing/test_fixtures/verify_signatures.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,20 @@

from pydantic import Field

from lean_spec.forks.lstar.containers.attestation import AggregatedAttestation
from lean_spec.forks.lstar.containers.block import (
SignedBlock,
)
from lean_spec.forks.lstar.containers.block.types import (
AggregatedAttestations,
AttestationSignatures,
)
from lean_spec.forks.lstar.containers.attestation import AggregatedAttestation, AttestationData
from lean_spec.forks.lstar.containers.block import SignedBlock
from lean_spec.forks.lstar.containers.block.types import AggregatedAttestations
from lean_spec.forks.lstar.containers.state import State
from lean_spec.forks.lstar.spec import LstarSpec
from lean_spec.types import AggregationBits, Boolean, ValidatorIndex
from lean_spec.types import (
AggregationBits,
Boolean,
ByteList512KiB,
Bytes32,
Checkpoint,
Slot,
ValidatorIndex,
)

from ..keys import XmssKeyManager
from ..test_types import BlockSpec
Expand Down Expand Up @@ -60,10 +63,6 @@ class VerifySignaturesTest(BaseConsensusFixture):

Supported operations:

- `{"operation": "drop_last_signature"}`: Remove the last entry
from the block's attestation_signatures list. Produces a signed
block whose signature-group count is one less than its
attestation count.
- `{"operation": "set_proposer_index", "value": int}`: Rewrite
Comment thread
tcoratger marked this conversation as resolved.
the block's proposer_index field. Use this to exercise the
validator-bounds check that the builder skips because its round-
Expand All @@ -72,6 +71,15 @@ class VerifySignaturesTest(BaseConsensusFixture):
first body attestation with one whose aggregation_bits carry no
set bit. Exercises the empty-participants check inside
signature verification.
- `{"operation": "corrupt_proof"}`: Replace the merged proof with
a short non-decodable blob. Exercises the Type-2 decode check.
- `{"operation": "append_phantom_attestation"}`: Add a body
attestation with no matching proof component. Exercises the
component count check between the body and the merged proof.
- `{"operation": "mutate_state_root"}`: Change a block field after
signing so the block root differs. Exercises the per-component
message binding that prevents reusing an honest proof under a
different message.

Tampered blocks bypass the builder's structural invariants. The
resulting fixture pins the exact rejection a client must raise when
Expand Down Expand Up @@ -153,16 +161,6 @@ def _apply_tamper(self, signed_block: SignedBlock) -> SignedBlock:
assert self.tamper is not None
operation = self.tamper.get("operation")

if operation == "drop_last_signature":
original = signed_block.signature.attestation_signatures.data
if not original:
raise ValueError("drop_last_signature requires at least one attestation signature")
truncated = AttestationSignatures(data=list(original[:-1]))
tampered_signatures = signed_block.signature.model_copy(
update={"attestation_signatures": truncated}
)
return signed_block.model_copy(update={"signature": tampered_signatures})

if operation == "set_proposer_index":
value = self.tamper.get("value")
if value is None:
Expand All @@ -185,4 +183,43 @@ def _apply_tamper(self, signed_block: SignedBlock) -> SignedBlock:
new_block = signed_block.block.model_copy(update={"body": new_body})
return signed_block.model_copy(update={"block": new_block})

if operation == "corrupt_proof":
# Replace the merged proof with a short non-decodable blob.
# Decoding the Type-2 envelope must fail before verification.
return signed_block.model_copy(
update={"proof": ByteList512KiB(data=b"\x00\x01\x02\x03")}
)

if operation == "append_phantom_attestation":
# Add a body attestation with no matching proof component.
# The proof binds one component per original attestation plus
# the proposer, so the body now claims more components than the
# proof carries.
body = signed_block.block.body
phantom_data = AttestationData(
slot=Slot(0),
head=Checkpoint(root=Bytes32(b"\x00" * 32), slot=Slot(0)),
target=Checkpoint(root=Bytes32(b"\x00" * 32), slot=Slot(0)),
source=Checkpoint(root=Bytes32(b"\x00" * 32), slot=Slot(0)),
)
phantom = AggregatedAttestation(
aggregation_bits=AggregationBits(data=[Boolean(True)]),
data=phantom_data,
)
new_attestations = AggregatedAttestations(data=[*body.attestations.data, phantom])
new_body = body.model_copy(update={"attestations": new_attestations})
new_block = signed_block.block.model_copy(update={"body": new_body})
return signed_block.model_copy(update={"block": new_block})

if operation == "mutate_state_root":
# Change a block field after signing so the block root differs.
# The proposer component's bound message no longer matches the
# recomputed block root, even though the signature is honest.
# This is the repackaging vector: an honest proof reused under
# a different message.
tampered_block = signed_block.block.model_copy(
update={"state_root": Bytes32(b"\xff" * 32)}
)
return signed_block.model_copy(update={"block": tampered_block})

raise ValueError(f"Unknown tamper operation: {operation!r}")
Loading
Loading