Skip to content

Implement StateTest JSON runner for official test vectors #77

@mw2000

Description

@mw2000

Summary

Build a test runner that parses and executes the official Ethereum execution test fixtures (EEST / ethereum-tests). This is the critical validation step that verifies eevm against thousands of official test vectors covering every opcode, gas rule, and edge case.

Specification

StateTest JSON format:

{
  "testName": {
    "env": { "currentCoinbase": "...", "currentNumber": "...", ... },
    "pre": { "address": { "balance": "...", "nonce": "...", "code": "...", "storage": {...} } },
    "transaction": { "sender": "...", "to": "...", "data": ["..."], "gasLimit": ["..."], ... },
    "post": {
      "Shanghai": [{ "hash": "0x...", "indexes": { "data": 0, "gas": 0, "value": 0 } }]
    }
  }
}

The runner must:

  1. Parse the JSON fixture
  2. Load pre state into Database.InMemory
  3. Set up Context.Block from env
  4. Execute the transaction (apply validation, intrinsic gas, then EVM execution)
  5. Compute the post-state root using MPT
  6. Compare against the expected hash in post

Implementation Guide

  1. Create test/support/state_test_runner.ex — parsing + execution harness
  2. Wire MPT state root computation:
    • Build account trie from all accounts in DB
    • Each account leaf: RLP.encode([nonce, balance, storage_root, code_hash])
    • Storage root: MPT of {keccak(slot) => RLP.encode(value)} for each account
  3. Create test/state_test_test.exs — parameterized test that loads fixtures and runs them
  4. Download fixtures: EEST release v5.4.0 from https://github.com/ethereum/execution-spec-tests/releases
  5. Start with simple fixtures: arithmetic, comparison, bitwise opcodes first, then gas, then system calls

Acceptance Criteria

  • Parses StateTest JSON format correctly
  • Loads pre-state into Database.InMemory
  • Executes transactions with full gas accounting
  • Computes post-state root via MPT
  • Passes basic arithmetic/comparison test fixtures
  • Reports pass/fail per test case with clear error messages
  • Can be run via mix test test/state_test_test.exs

Reference

Metadata

Metadata

Assignees

No one assigned

    Labels

    coreCore EVM functionalitytestingTest infrastructure and validation

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions