Skip to content

Conversation

@anvacaru
Copy link
Contributor

@anvacaru anvacaru commented Dec 11, 2025

Refactor the semantics of driver.md and remove the rules associated with chaining multiple tests.

  • Simplified run rules to process a single test entry instead of chaining multiple entries
  • Removed failure String syntax and associated rules; since only one test is executed at a time and the test harness manages test names externally, this is no longer needed
  • Removed TESTID propagation from run, process, and check commands
  • Removed unused output-success-java.json file
  • Updated output-success-{haskell,llvm}.json files.

@anvacaru anvacaru marked this pull request as ready for review December 11, 2025 13:41
@ehildenb
Copy link
Member

Did you confirm that breaking a vmtest and a bchaintest manually and the test-harness correctly finds the breakage?

anvacaru and others added 2 commits December 11, 2025 22:15
Co-authored-by: Everett Hildenbrandt <everett.hildenbrandt@gmail.com>
@anvacaru
Copy link
Contributor Author

Did you confirm that breaking a vmtest and a bchaintest manually and the test-harness correctly finds the breakage?

Yes, I modified the gasUsed blockheader value in VMTests/vmArithmeticTest/add.json and they fail as expected.

[1] anvacaru@desktop kevm-pyk$ uv run -- pytest src/tests/integration --verbose --durations=0 --dist=worksteal --maxfail=3 --numprocesses=2 -k 'test_conformance.py and (15_tstoreCannotBeDosd.json or add.json)
=================================================================================== test session starts ===================================================================================
src/tests/integration/test_conformance.py::test_bchain[stZeroKnowledge/pointAdd.json]
src/tests/integration/test_conformance.py::test_vm[vmArithmeticTest/add.json]
[gw0] [ 25%] FAILED src/tests/integration/test_conformance.py::test_vm[vmArithmeticTest/add.json]
src/tests/integration/test_conformance.py::test_bchain[stZeroKnowledge/pointMulAdd.json]
[gw1] [ 50%] PASSED src/tests/integration/test_conformance.py::test_bchain[stZeroKnowledge/pointAdd.json]
src/tests/integration/test_conformance.py::test_bchain[Cancun/stEIP1153-transientStorage/15_tstoreCannotBeDosd.json]
[gw0] [ 75%] PASSED src/tests/integration/test_conformance.py::test_bchain[stZeroKnowledge/pointMulAdd.json]
[gw1] [100%] FAILED src/tests/integration/test_conformance.py::test_bchain[Cancun/stEIP1153-transientStorage/15_tstoreCannotBeDosd.json]

@automergerpr-permission-manager automergerpr-permission-manager bot merged commit 1e4dda8 into master Dec 12, 2025
23 of 24 checks passed
@automergerpr-permission-manager automergerpr-permission-manager bot deleted the remove-GST-chaining branch December 12, 2025 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants