Skip to content

test: update test runner to verify signatures#310

Merged
MegaRedHand merged 1 commit intomainfrom
update-spectest-runner
Apr 20, 2026
Merged

test: update test runner to verify signatures#310
MegaRedHand merged 1 commit intomainfrom
update-spectest-runner

Conversation

@MegaRedHand
Copy link
Copy Markdown
Collaborator

@MegaRedHand MegaRedHand commented Apr 20, 2026

This PR adds some changes that were mistakenly removed from #304. It updates the test runner to run the full attestation flow (signature verification included), which removes some verification-less paths that are no longer necessary. Before, we weren't doing signature verification, and had a wrong implementation for aggregated proofs processing.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Apr 20, 2026

Greptile Summary

This PR restores changes accidentally dropped from #304, wiring the fork-choice spec-test runner to execute the full attestation flow — including XMSS signature verification — for both individual (attestation) and aggregated (gossipAggregatedAttestation) steps. As a result, the test-only bypass function on_gossip_attestation_without_verification is removed from store.rs, and deser_xmss_hex is consolidated into test-fixtures so both test crates share the same deserializer.

Confidence Score: 5/5

Safe to merge; all remaining findings are P2 style suggestions in test utilities only.

The production code change is a clean removal of a test-bypass function and a minor borrow-ordering refactor with no behavioral difference. The only flagged item is a low-context panic message in a test helper.

No files require special attention.

Important Files Changed

Filename Overview
crates/blockchain/src/store.rs Removes test-only on_gossip_attestation_without_verification bypass; refactors on_gossip_aggregated_attestation to read stats before consuming the proof (borrow-checker fix); updates doc comment to reflect the stored data model.
crates/blockchain/tests/forkchoice_spectests.rs Test runner now builds real SignedAttestation and SignedAggregatedAttestation objects with actual signatures from fixtures, replacing the old signature-bypass codepath.
crates/blockchain/tests/signature_types.rs Removes duplicate deser_xmss_hex definition (now imported from super::common); no functional change to verify-signatures test infrastructure.
crates/blockchain/tests/types.rs Adds is_aggregator to ForkChoiceStep, promotes signature to Option<XmssSignature>, and introduces ProofStepData/HexByteList for aggregated attestation fixtures.
crates/common/test-fixtures/src/lib.rs Adds the canonical deser_xmss_hex helper migrated from signature_types.rs, enabling deduplication across test crates.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: crates/blockchain/tests/types.rs
Line: 91-96

Comment:
**Panic on invalid hex in test fixture**

`From<HexByteList>` calls `expect` on hex decoding, so a malformed `0x…` string in a fixture will produce an opaque panic rather than a structured serde/test error. The context string `"invalid hex in proof data"` gives no indication of which step or test case failed.

This is test code, so a panic is not catastrophic, but surfacing the offending string in the message would make fixture authoring failures much easier to diagnose.

```suggestion
        hex::decode(stripped)
            .unwrap_or_else(|_| panic!("invalid hex in proof data: {:?}", value.data))
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "test: update test runner to verify signa..." | Re-trigger Greptile

@github-actions
Copy link
Copy Markdown

🤖 Kimi Code Review

Overall Assessment: Good refactoring that removes unsafe test bypasses and improves aggregation logic. The move from per-validator to per-attestation-data storage is correct for consensus, but verify the store implementation aggregates participant bits rather than overwriting.


Critical: Aggregation Storage Logic

crates/blockchain/src/store.rs:657

The change stores one entry per unique attestation data rather than per validator. Ensure Store::insert_new_aggregated_payload aggregates participant bits when an entry for the same hashed data already exists. If it simply overwrites, attestations from disjoint aggregators will be lost, breaking fork choice weight calculations.

Suggested verification: Check that insert_new_aggregated_payload performs a bitwise OR on the participant bitfield if an entry exists, or document that the store backend handles this internally.


Safety: Removal of Verification Bypass

crates/blockchain/src/store.rs:689-718 (deleted)

Removing on_gossip_attestation_without_verification eliminates a critical attack surface. Positive: This prevents accidental production use of unverified attestations.

crates/blockchain/tests/forkchoice_spectests.rs:97-118

The test harness now uses real signature verification. Ensure the SKIP_TESTS list (referenced at line 28) includes all fixtures lacking valid XMSS signatures, or tests will spuriously fail. The updated skip message ("see SKIP_TESTS comment") is vague—document specific reasons per test (e.g., "missing aggregator signatures").


Correctness: Aggregated Attestation Construction

crates/blockchain/tests/forkchoice_spectests.rs:124-137

The SignedAggregatedAttestation is constructed with only data and proof fields. Verify the struct definition doesn't require a separate signature field for the outer wrapper (distinct from the proof's internal signature). If the consensus spec expects a signed outer envelope, this construction is incomplete.


Error Handling

crates/blockchain/tests/types.rs:86

impl From<HexByteList> for Vec<u8> {
    fn from(value: HexByteList) -> Self {
        let stripped = value.data.strip_prefix("0x").unwrap_or(&value.data);
        hex::decode(stripped).expect("invalid hex in proof data")  // Panic risk
    }
}

Issue: expect in deserialization will panic on malformed fixtures. Use map_err to return a proper deserialization error instead:

hex::decode(stripped).map_err(|e| serde::de::Error::custom(format!("invalid hex: {e}")))?

However, since this is test-only code and the From trait doesn't allow custom error types, consider changing this to a try_into pattern or accepting the panic risk with a comment noting it requires valid fixtures.


Code Quality

crates/blockchain/tests/forkchoice_spectests.rs:147-160

Excellent deduplication using assert_step_outcome. The generic bounds <T, E: std::fmt::Debug> are correct for test error reporting.

crates/blockchain/src/store.rs:650-654

Good practice reading fields (num_participants, target_slot, etc.) before the aggregated.proof move at line 657. This prevents use-after-move bugs and avoids unnecessary cloning.


Minor: Documentation

crates/blockchain/src/store.rs:605

The updated comment clarifies the aggregation strategy, but explicitly mention that participant bits are aggregated (OR'd) if an entry exists, not just "carried in the proof itself."


Summary

  1. Verify insert_new_aggregated_payload aggregates participant bits correctly ( Item 1 )
  2. Confirm SKIP_TESTS covers all fixtures without valid signatures ( Item 2 )
  3. Optional: Replace expect with fallible deserialization in HexByteList ( Item 4 )

The signature verification wiring and removal of bypass functions significantly improves consensus safety.


Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt

Comment on lines +91 to +96
impl From<HexByteList> for Vec<u8> {
fn from(value: HexByteList) -> Self {
let stripped = value.data.strip_prefix("0x").unwrap_or(&value.data);
hex::decode(stripped).expect("invalid hex in proof data")
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Panic on invalid hex in test fixture

From<HexByteList> calls expect on hex decoding, so a malformed 0x… string in a fixture will produce an opaque panic rather than a structured serde/test error. The context string "invalid hex in proof data" gives no indication of which step or test case failed.

This is test code, so a panic is not catastrophic, but surfacing the offending string in the message would make fixture authoring failures much easier to diagnose.

Suggested change
impl From<HexByteList> for Vec<u8> {
fn from(value: HexByteList) -> Self {
let stripped = value.data.strip_prefix("0x").unwrap_or(&value.data);
hex::decode(stripped).expect("invalid hex in proof data")
}
}
hex::decode(stripped)
.unwrap_or_else(|_| panic!("invalid hex in proof data: {:?}", value.data))
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/blockchain/tests/types.rs
Line: 91-96

Comment:
**Panic on invalid hex in test fixture**

`From<HexByteList>` calls `expect` on hex decoding, so a malformed `0x…` string in a fixture will produce an opaque panic rather than a structured serde/test error. The context string `"invalid hex in proof data"` gives no indication of which step or test case failed.

This is test code, so a panic is not catastrophic, but surfacing the offending string in the message would make fixture authoring failures much easier to diagnose.

```suggestion
        hex::decode(stripped)
            .unwrap_or_else(|_| panic!("invalid hex in proof data: {:?}", value.data))
```

How can I resolve this? If you propose a fix, please make it concise.

@github-actions
Copy link
Copy Markdown

🤖 Claude Code Review

Now I have everything I need. Here is the full review:


PR #310 Review: Update test runner to verify signatures

Overview

This PR restores changes that were accidentally dropped from PR #304. It wires real XMSS signature verification into the fork choice spec test runner, replacing the now-deleted on_gossip_attestation_without_verification() bypass. It also deduplicates the step-outcome assertion logic and consolidates deser_xmss_hex into the shared test-fixtures crate.


Correctness

Move-before-borrow fix in store.rs (lines 650–667): The change from aggregated.proof.clone() to extracting fields first and then moving is correct. Previously, aggregated was partially moved (.proof was cloned and passed in), but then aggregated.data.slot etc. were accessed after the call — which could have compiled only because .clone() was used, not a full move. Now the code explicitly extracts all needed values before the move, making ownership clear and avoiding the unnecessary clone. This is idiomatic and correct.

assert_step_outcome helper: The match arms correctly handle all four cases. (Ok(_), true) and (Err(_), false) collapse to Ok(()) via the wildcard arm, which is correct semantics: both are the "expected" outcomes that should not fail the test.

deser_opt_xmss_hex: The Wrap newtype trick is idiomatic serde and correctly handles None when the field is absent in the fixture JSON.


Potential Issues

SKIP_TESTS message references a non-existent comment. After this PR, SKIP_TESTS is &[] (empty), so the skip branch is dead code. The message "Skipping {stem} (see SKIP_TESTS comment)" references a comment that doesn't exist. This isn't a bug now, but when tests are added to SKIP_TESTS later, developers will be confused. Consider either removing the dead branch or adding a placeholder comment at the SKIP_TESTS definition explaining what future entries would mean.

HexByteList::From panics on invalid fixture data. The hex::decode(...).expect(...) in the From<HexByteList> impl will panic rather than produce a useful test failure. For test code this is acceptable since fixture corruption should surface loudly, but the expect message "invalid hex in proof data" could be more specific (e.g., include the offending string). Minor quality-of-life issue only.

ByteList::try_from panic. In the gossipAggregatedAttestation handler:

let proof_data = ByteList::try_from(proof_bytes)
    .expect("aggregated proof data fits in ByteListMiB");

If a fixture ever carries a proof larger than the ByteListMiB limit, this panics rather than failing the test with assert_step_outcome. Again acceptable for test code, but worth noting.

is_aggregator defaults silently to false. The unwrap_or(false) default for is_aggregator in the gossipAttestation branch is correct for the current fixtures, but future fixtures that omit the field will silently test the non-aggregator path. A doc comment on the is_aggregator field in ForkChoiceStep stating the default semantics would help fixture authors.


Code Quality

DRY improvement is significant. The three duplicated 15-line match (result.is_ok(), step.valid) blocks are cleanly replaced by a single generic helper. The helper's type signature is clear and the error messages are faithful to the originals.

deser_xmss_hex consolidation into test-fixtures is the right move — it belongs with other fixture deserialization helpers, not in a test-specific module. The implementation is unchanged and correct.

Doc comment update on on_gossip_aggregated_attestation accurately reflects the new storage behavior (one entry per attestation data, participants carried in the proof). This is a clear improvement over the old, now-inaccurate comment.

Removed // Verify aggregated proof signature inline comment — good; the verification is implicit in the code path and the comment added no value.


Summary

The PR is clean, well-scoped, and achieves its goal. The borrow fix (move without clone) is a genuine correctness/performance improvement. The test helper deduplication is a meaningful DRY win. No logic regressions are visible.

The SKIP_TESTS dead-code message is the only thing worth fixing before merge; the rest are minor.


Automated review by Claude (Anthropic) · sonnet · custom prompt

@github-actions
Copy link
Copy Markdown

🤖 Codex Code Review

Findings:

  1. [forkchoice_spectests.rs:75](/home/runner/work/ethlambda/ethlambda/crates/blockchain/tests/forkchoice_spectests.rs#L75) and [forkchoice_spectests.rs:91](/home/runner/work/ethlambda/ethlambda/crates/blockchain/tests/forkchoice_spectests.rs#L91) still hardcode is_aggregator = false on every on_tick, but [forkchoice_spectests.rs:106](/home/runner/work/ethlambda/ethlambda/crates/blockchain/tests/forkchoice_spectests.rs#L106) now routes "attestation" steps through [store.rs:535](/home/runner/work/ethlambda/ethlambda/crates/blockchain/src/store.rs#L535), which only persists the signature when is_aggregator is true at [store.rs:579](/home/runner/work/ethlambda/ethlambda/crates/blockchain/src/store.rs#L579). That means raw attestation steps will never be aggregated at interval 2 in this harness, so they no longer affect fork choice unless the fixture also injects a separate "gossipAggregatedAttestation" step. For a fork-choice conformance test, that is a real coverage hole and can hide LMD GHOST / justification regressions.

  2. [types.rs:91](/home/runner/work/ethlambda/ethlambda/crates/blockchain/tests/types.rs#L91) and [forkchoice_spectests.rs:123](/home/runner/work/ethlambda/ethlambda/crates/blockchain/tests/forkchoice_spectests.rs#L123) use expect(...) for proofData decoding. A malformed or oversized fixture now panics the whole test binary instead of returning a scoped datatest failure for that case. This is test-only, but it makes the harness much harder to debug; proofData should be deserialized fallibly and bubbled up as a normal test error.

The store.rs hunk itself looks fine; it’s effectively a cleanup plus one less clone.

I couldn’t run the Rust test target here because cargo needs writable ~/.cargo/~/.rustup state and a git fetch for lean-multisig, which this environment blocks.


Automated review by OpenAI Codex · gpt-5.4 · custom prompt

@MegaRedHand MegaRedHand merged commit cb7151a into main Apr 20, 2026
7 checks passed
@MegaRedHand MegaRedHand deleted the update-spectest-runner branch April 20, 2026 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants