Conformance Testing

This guide details how to test your changes to Agave for conformance with the Solana protocol.

This guide is intended for all contributors making changes to Agave who wish to ensure their changes:

• Do not break consensus
• Do not introduce a regression
• Produce the expected results for the components changed, beyond any included unit tests

Conformance testing is designed to leverage artifacts - such as fixtures and ledgers - to ensure the end-to-end protocol implementation for a given component still produces the expected result for a given set of inputs. The specific details of how each form of conformance testing works are provided below, as well as the necessary instructions to run them yourself.

Thank you to the Firedancer team for spearheading this effort and enabling it for Agave contributions.

Fixture-Based Testing

First identify the entrypoints you wish to invoke with test vectors (fixtures). This will depend on which Agave components you've modified, and can include more than one.

The table below details each vector entrypoint and its associated coverage.

Entrypoint	Responsibility	Primary Crates Exercised
sol_compat_gossip_decode_v1	Deserialize a binary gossip wire message and decode it into structured effects; validates CRDS/gossip protocol parsing.	solana-gossip
sol_compat_shred_parse_v1	Ingest serialized shreds through the blockstore ingest/recovery pipeline and replay validation; exercises shred parsing and blockstore-owned handling.	solana-ledger
sol_compat_block_execute_v1	Reconstruct block state, commit the block's transactions through the Bank, track cost, and return bank and leader-schedule effects.	solana-runtime, solana-accounts-db
sol_compat_txn_cost_v1	Compute cost-model accounting for a transaction (signature, write-lock, data-bytes, execution, and loaded-accounts-data-size costs).	solana-cost-model, solana-runtime-transaction
sol_compat_txn_execute_v1	Execute a full transaction through the real Bank load-and-execute path (fees, nonce, rent, account loading, program execution). Runtime-level transaction processing.	solana-runtime, solana-svm, solana-accounts-db
sol_compat_svm_txn_execute_v1	Execute a sanitized transaction message (all instructions) directly through the SVM invoke context — no bank fee/nonce/loader machinery. SVM-level message processing.	solana-svm, solana-program-runtime
sol_compat_instr_execute_v1	Execute a single instruction against the SVM via InvokeContext::process_message, resolving the builtin/BPF program from the program cache.	solana-program-runtime, solana-bpf-loader-program, solana-system-program, agave-precompiles
sol_compat_vm_serialize_execute_v1	Serialize an instruction's program input parameters into VM memory; reports the serialized-memory hash, input memory regions, and per-account metadata.	solana-program-runtime
sol_compat_vm_syscall_execute_v1	Set up an sBPF VM and invoke a single syscall, checking memory mapping, CU consumption, and post-syscall effects.	solana-syscalls, solana-program-runtime, solana-sbpf
sol_compat_elf_loader_v1	Load and verify an sBPF program ELF under a program-runtime environment; reports rodata hash, entry PC, text section layout, and call destinations.	solana-sbpf, solana-syscalls

The rest of this section walks through running fixtures for one entrypoint, using the ELF loader (sol_compat_elf_loader_v1) as the worked example. The same steps apply to every entrypoint — only the category name changes.

Getting the Test Vectors

Fixtures live in the test-vectors repository, grouped by entrypoint into per-category directories (elf_loader/, instr/, txn/, ...), each with a fixtures/ directory of .fix files. A .fix is a Protobuf-encoded <Category>Fixture holding the recorded input and expected output for a single entrypoint invocation.

Until fixtures are published as release tarballs, clone the repository and point a runner directly at the relevant <category>/fixtures/ directory:

git clone https://github.com/firedancer-io/test-vectors.git

Running the Vectors

Each entrypoint has a runner binary in the solana-svm-conformance crate (dev-bins workspace), named test_exec_<category> — here, test_exec_elf_loader. It decodes each .fix, runs it through Agave, and compares the result against the expected output.

Build the desired runner:

cargo build --manifest-path dev-bins/Cargo.toml -p solana-svm-conformance

Run it against its fixtures:

dev-bins/target/debug/test_exec_elf_loader -- test-vectors/elf_loader/fixtures/*.fix

Each fixture prints OK: <file> or FAIL: <file> (with the expected and actual effects on failure); the process exits with the number of failures, so 0 means every vector passed. For the tally alone:

dev-bins/target/debug/test_exec_elf_loader -- test-vectors/elf_loader/fixtures/*.fix \
  | grep -c '^OK:'