-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(fw): run evm blocktest on generated fixtures #325
feat(fw): run evm blocktest on generated fixtures #325
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks very good! Just a few comments to keep in mind.
If --evm-test-bin is not specified but --do-evm-test is specified, then try to use evm_bin as evm_test_bin.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes after discussing with Mario.
This is achieved by stripping 'test_' from the test module and test function names in the output paths used for fixtures. The evm dump dir keeps the prefix.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM :)
🗒️ Description
This PR optionally enables the execution of
evm blocktest
on each generated JSON fixture file. The check is ran as a function in theFixtureCollector
in a module-scope pytest fixture (so is not executed per test function or per test-case).If
--t8n-dump-dir
is additionally specified, the output of theevm blocktest
command is also written there along with averify_fixtures.sh
script to re-run the command.There's a couple of other changes to the t8n-dump directories:
Examples:
🔗 Related Issues
evm blocktest
within tox #237 (just need to add the calls with relevant command-line arguments in tox or ci/cd).evm
binaries and compare results #185 partially redundant.✅ Checklist
mkdocs serve
locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.