feat: verify-stark guest library and SDK integration

[stephenh-axiom-xyz]

Resolves INT-6548, INT-6194, INT-6641.

High-Level Overview

This PR adds end-to-end verify-stark deferral support in three stages:

1. Moves the verify-stark circuit/prover code into a dedicated crate.
2. Adds a verify-stark guest library and extension wiring.
3. Adds deferral-aware proof/baseline/verification support in openvm-verify-stark-host and the SDK.

The branch also includes follow-up correctness fixes (Merkle proof shape, output hashing alignment, def_vk_commit source) and updated tests.

1) Move The Verify-Stark Circuit Crate

What changed

• New crate: guest-libs/verify-stark/circuit (openvm-verify-stark-circuit).
• Moved deferred-verify circuit logic out of crates/continuations:
  • old: crates/continuations/src/circuit/deferral/verify/*
  • old: crates/continuations/src/prover/deferral/verify/*
  • new: guest-libs/verify-stark/circuit/src/* (AIRs, tracegen, prover, tests).
• crates/continuations refactor to support the move:
  • deferral modules flattened from deferral/aggregation/{hook,inner} to deferral/{hook,inner}.
  • added DeferralCircuitProver trait as the integration interface for external deferral circuits.
  • exposed shared utilities/constants needed by the moved crate (circuit::utils, root::NUM_DIGESTS_IN_VK_COMMIT, etc.).

Reviewer entry points

• guest-libs/verify-stark/circuit/src/lib.rs
• guest-libs/verify-stark/circuit/src/prover/mod.rs
• crates/continuations/src/prover/deferral/mod.rs
• crates/continuations/src/prover/mod.rs

2) Add Verify-Stark Guest Library + Extension Support

What changed

• New guest crate: guest-libs/verify-stark/guest (openvm-verify-stark-guest).
  • Adds guest-facing API to fetch/validate deferred proof output:
    • get_proof_output
    • verify_proof_output
• New extension helpers in the circuit crate:
  • verify_stark_deferral_fn
  • get_raw_deferral_results
  • get_deferral_state
• Workspace and dependency split:
  • replaced old openvm-verify-stark path dependency with:
    • openvm-verify-stark-circuit
    • openvm-verify-stark-guest
• Deferral extension updates:
  • DeferralExtension now carries def_vk_commits (serialized, transpiler-facing).
  • fns remains runtime-only (serde(skip)).
• SDK config wiring:
  • SdkVmConfig now supports optional deferral extension in transpiler + prover extension setup.
• VM deferral state serialization:
  • DeferralState and InputMapVal derive serde traits.

Reviewer entry points

• guest-libs/verify-stark/guest/src/lib.rs
• guest-libs/verify-stark/circuit/src/extension.rs
• extensions/deferral/circuit/src/extension/mod.rs
• crates/sdk-config/src/lib.rs
• crates/vm/src/arch/deferral.rs

3) Add Deferrals To openvm-verify-stark-host And SDK

Host-side verification (crates/verify)

• NonRootStarkProof now optionally includes deferral_merkle_proofs (encoded/decoded with proof bytes).
• New deferral.rs adds DeferralMerkleProofs and path verification logic.
• VerificationBaseline now includes expected_def_vk_commit: Option<Digest>.
• verify_vm_stark_proof_decoded now enforces deferral invariants when deferrals are expected:
  • valid deferral flag,
  • matching deferral hook VK commit,
  • presence of deferral Merkle proofs,
  • verified initial/final deferral path roots.
• Expanded VerifyStarkError with deferral-specific failure modes.

SDK proving flow (crates/sdk)

• New deferral prover modules:
  • sdk/src/prover/deferral/mod.rs
  • sdk/src/prover/deferral/circuit.rs
  • sdk/src/prover/deferral/merkle.rs
• AggProver split into VM/deferral/mixed operations:
  • prove_vm, prove_def, prove_mixed, and proof-type-aware wrap_proof.
• GenericSdk and StarkProver now support optional deferral path proving:
  • GenericSdk::with_deferral_prover(...)
  • prove(..., def_inputs: &[DeferralInput])
• Proving now attaches deferral_merkle_proofs to produced NonRootStarkProof when deferrals are enabled.
• Baseline generation now emits expected_def_vk_commit for verifier-side checks.
• Serialized proof type updated (VersionedNonRootStarkProof) to carry optional deferral proof bytes.

Reviewer entry points

• crates/verify/src/lib.rs
• crates/verify/src/deferral.rs
• crates/sdk/src/prover/stark.rs
• crates/sdk/src/prover/agg.rs
• crates/sdk/src/prover/deferral/mod.rs
• crates/sdk/src/types.rs

Test Coverage And Follow-Up Fixes

• Added SDK integration test for end-to-end verify-stark deferral flow:
  • crates/sdk/src/tests.rs::test_verify_stark_deferral
• Added dedicated tests in moved circuit crate:
  • guest-libs/verify-stark/circuit/src/tests.rs
• Continuations tests refactored to use dummy deferral proof helpers; legacy tests/verify.rs removed.
• Follow-up fixes included in this branch:
  • deferral output commit AIR/trace alignment with extension output format,
  • full-length deferral Merkle proof handling,
  • def_vk_commit baseline source corrected to use aggregation prover DAG commits.

[github-actions]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.

[stephenh-axiom-xyz]

ATTN: @shuklaayush @jonathanpwang. We have a ticket to change this, but wanted to note that an AIR is being changed

[github-actions]

Claude finished @stephenh-axiom-xyz's task in 4m 0s —— View job

---

Code Review: feat: verify-stark guest library and SDK integration

• Gather context and diff
• Review circuit crate move (stage 1)
• Review guest library + extension wiring (stage 2)
• Review host verification + SDK integration (stage 3)
• Review tests
• Post final review

---

Overall this is a well-structured PR that cleanly separates the verify-stark circuit, guest library, and SDK integration across three logical stages. The abstraction boundaries are sensible (especially DeferralCircuitProver trait and DeferralProof::Absent variant), and the end-to-end test provides good coverage. Below are findings organized by severity.

---

Medium: Trailing singleton Absent in reduce_def_round not promoted in depth

crates/sdk/src/prover/agg.rs:354-362

When a trailing singleton is Absent, it's passed through unchanged:

None => {
    let out = match a {
        DeferralProof::Present(p) => DeferralProof::Present(
            prover.agg_prove::<E>(&[p], kind, ProofsType::Deferral, None)?,
        ),
        absent => absent,  // depth is NOT incremented
    };
    next.push(out);
}

But for Present trailing singletons, a wrapping proof is generated (which increments depth via the circuit). And when two Absents are paired, depth is explicitly incremented on line 348. This means a trailing Absent keeps its original depth, creating a depth mismatch with its sibling in the next round. If this code path is actually reachable (odd number of inputs), the debug_assert_eq!(pvs0.depth, pvs1.depth) on line 336 would fire on the next round.

Consider incrementing depth for trailing Absent singletons:

DeferralProof::Absent(pvs) => DeferralProof::Absent(DeferralPvs {
    depth: pvs.depth + F::ONE,
    ..pvs
}),

Or document that odd-length input is not expected.

Fix this →

---

Medium: DeferralCircuitProver::prove panics instead of returning Result

crates/continuations/src/prover/deferral/mod.rs:14

pub trait DeferralCircuitProver<SC: StarkProtocolConfig> {
    fn get_vk(&self) -> Arc<MultiStarkVerifyingKey<SC>>;
    fn prove(&self, input_bytes: &[u8]) -> Proof<SC>;
}

The trait's prove returns Proof<SC> (infallible), but the implementation in guest-libs/verify-stark/circuit/src/prover/mod.rs:254-258 calls .unwrap() on both decode and prove:

fn prove(&self, input_bytes: &[u8]) -> Proof<SC> {
    let non_root_proof = NonRootStarkProof::decode_from_bytes(input_bytes).unwrap();
    self.prover
        .prove_no_def::<E>(non_root_proof.inner, &non_root_proof.user_pvs_proof)
        .expect("DeferredVerifyProver::prove_no_def failed")
}

Since this trait will be implemented by external circuit provers, consider returning Result<Proof<SC>> to allow callers to handle failures gracefully instead of panicking.

---

Medium: EvmProver::prove does not support deferrals

crates/sdk/src/prover/evm.rs:47-101

EvmProver::prove still takes only StdIn with no def_inputs parameter and always passes ProofsType::Vm to wrap_proof. If a user creates an SDK with deferrals enabled and then tries the EVM proof path, deferrals will silently be ignored (no deferral proofs generated, no mixed aggregation, no merkle proofs attached). This should either:

• Accept def_inputs and mirror StarkProver::prove's deferral logic, or
• Assert/error that deferrals are not enabled when using the EVM path

The TODO [INT-6241] in the test suggests this is known, but it would be good to add a runtime check to prevent silent misuse.

---

Low-Medium: Misleading AIR comment about next_f_idx increment

guest-libs/verify-stark/circuit/src/output/air.rs:85

Comment says "we also constrain that next_f_idx increments by VALS_IN_DIGEST" but the actual constraint on line 99 increments by 1:

builder.when_transition().assert_eq(local.next_f_idx + AB::Expr::ONE, next.next_f_idx);

Each row processes VALS_IN_DIGEST values, but next_f_idx increments by 1 per row (it counts rows, not individual values). The constraint is correct — the comment is misleading. In a ZK proof system, comment/constraint mismatches are a review hazard.

---

Low-Medium: Deferral flag verification could be simplified

crates/verify/src/lib.rs:223-229

let flag = deferral_flag.as_canonical_u32();
if flag != 0 && flag != 2 {
    return Err(VerifyStarkError::InvalidDeferralFlag(deferral_flag));
}
if flag == 0 {
    return Err(VerifyStarkError::DeferralFlagNotSet);
}

Inside the if let Some(expected_def_vk_commit) block, deferrals are expected, so flag must be 2. This two-step check could be simplified to:

if flag != 2 {
    return Err(if flag == 0 {
        VerifyStarkError::DeferralFlagNotSet
    } else {
        VerifyStarkError::InvalidDeferralFlag(deferral_flag)
    });
}

This is functionally equivalent but clearer about the intent.

---

Low: .unwrap() calls on potentially untrusted data

Several .unwrap() calls in code paths that handle externally-provided data:

Location	Context
guest-libs/verify-stark/circuit/src/extension.rs:36	decode_from_bytes(...).unwrap() on proof bytes
guest-libs/verify-stark/circuit/src/extension.rs:89	.last().unwrap() on transcript log
crates/verify/src/deferral.rs:70	usize::try_from(...).unwrap() on label_to_index result
crates/sdk/src/stdin.rs:83	decode_from_bytes(input).unwrap() in DeferralInput::into_inputs

For the verifier path (crates/verify), panics on malformed input are denial-of-service vectors. Consider replacing with ? / ok_or(...) where feasible.

---

Low: TODO left in production code — hash function choice

guest-libs/verify-stark/circuit/src/extension.rs:96 and crates/sdk/src/prover/deferral/circuit.rs:93

// TODO[INT-6415]: hash slice, not compress
let input_commit = poseidon2_compress_with_capacity(ts_commit, cached_commit).0;

Using compress vs hash_slice has different security properties (domain separation, variable-length input handling). This appears in two places. Flagging in case this should be resolved before merge.

---

Nit: #[cfg(all(test, feature = "cuda"))] for tests

guest-libs/verify-stark/circuit/src/lib.rs:42-43

#[cfg(all(test, feature = "cuda"))]
mod tests;

Tests are gated behind the cuda feature. This means CI without CUDA will never run these tests. Is this intentional? The test file at guest-libs/verify-stark/circuit/src/tests.rs does use cfg_if to select CPU vs GPU provers, suggesting it could work without CUDA. Consider #[cfg(test)] to ensure CPU-mode test coverage.

---

Architecture: Clean design decisions

1. DeferralCircuitProver trait — nice abstraction allowing external deferral circuits to plug into the continuations framework without circular dependencies.

2. DeferralProof::Absent variant — elegantly handles the case where no deferrals exist for a circuit index, allowing the aggregation tree to propagate computed PVs without generating dummy proofs.

3. Merkle proof verification (crates/verify/src/deferral.rs) — the skip_depth approach with zero-padded proofs is clean and avoids special-casing at the aggregation level.

4. Guest library (guest-libs/verify-stark/guest/src/lib.rs) — minimal API surface (get_proof_output, verify_proof_output). Clean separation between guest-facing and host-facing code.

5. End-to-end test (crates/sdk/src/tests.rs::test_verify_stark_deferral) — thorough 10-step test covering the full flow from fibonacci proof through deferral to final verification.

---

Summary

Severity	Count	Key Items
Medium	3	Trailing Absent depth mismatch; prove trait panics; EVM path ignores deferrals
Low-Medium	2	Misleading AIR comment; deferral flag check simplification
Low	2	Unwrap calls on untrusted data; TODO hash function choice
Nit	1	Tests gated behind cuda feature

The most actionable items are the Absent depth tracking in reduce_def_round and adding a guard/assertion in the EVM prover path when deferrals are configured.

Note: The previous review's top finding (from_pk dropping def_hook_commit for leaf prover) appears to have been fixed — from_pk now correctly passes def_hook_commit through.