feat(fw): verkle pydantic rebase and t8n changes

[spencer-tb]

🗒️ Description

Adds specific changes from #465 & #485, including a rebase onto main after the pydantic PR #486.

Usage

Use the following geth branch gballet/t8n-verkle-exec to *both fill and execute tests.

Fill tests with the following commands (until we update the framework fully).

fill --fork ShanghaiToPragueAtTime32 -k "blockchain" --output fixtures-verkle-0 -n 4 -v
fill --fork Prague -k "verkle" --output fixtures-verkle-0 -n 4 -v

Execute tests using the consume rlp eest based hive simulator. Run the following command after generating the tests:

consume rlp --input=fixtures-verkle-0/ -v

---

Future Todos

STILL TO BE UPDATED

Tweaks

• Refine pydantic VerkleTree model and its integration within the TransitionToolOutput model.
• Add state test generation. Currently only blocktest and hive blocktest fixtures can be generated.

Fixture Validation

• Utilize the evm verkle tree-keys command to convert the post state expected allocation to its verkle tree representation. Compare these values with those from the final verkle tree output from t8n. Example subcommand usage below for an idea of what happens when calling the command:
• Add the witness values from evm t8n ... --output.witness to each block in the fixture, furthermore utilize during filling to backtest the vkt keys and values against the witness generated for each block.
• Add the witness with there respective key/values to the t8n debug dump dir.

Basic Fixture Execution

• Pass all appropriate generated fixtures in the consume rlp simulator.

The below errors all occur on the first verkle block, and look to be from the same cause:

• Pass the execution from the generated fixture within the evm blocktest command. Note the following error below. This can be ran within the same directory as fill.
• Pass the execution of the generated fixture within the hive pyspec simulator. Currently it assumes that we are using a full node. Blocks are executed via the engine api using engine_newPayloadV2, against the dockerized client under test. Currently we get a similar error. But from the engine new payload method.
  This can be ran on the following branch - https://github.com/spencer-tb/hive/tree/verkle/pyspec using this command below. We are using the kaustinan-with-shapella geth branch.

./hive --sim ethereum/pyspec --client go-ethereum --client-file "configs/verkle.yaml" --docker.output --docker.nocache pyspec --sim.loglevel 5 --loglevel 5

Further Fixture Generation and Execution

Once we are passing the basic fixture generation we should add the verkle transition for all tests.

Furthermore, add the filling and exection of all these tests starting from Prague. Note the input alloc to t8n will no longer exist in this case so we should first convert it using the evm verkle tree-keys subcommand to VKT reprenstation.

🔗 Related Issues & PRs

None

✅ PR Checklist

• All: Set appropriate labels for the changes.
• All: Considered squashing commits to improve commit history.
• All: Added an entry to CHANGELOG.md.
• All: Considered updating the online docs in the ./docs/ directory.
• Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
• Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
• Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.