-
Notifications
You must be signed in to change notification settings - Fork 471
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make VMTests pass again (both concrete and symbolic) #1163
Comments
Hi @feliam I was also looking into this today while checking #1167, most tests that do not pass are due to gas mismatch, I disassembled one example test, compared opcodes that come from pyevmasm and fees against to what I saw from my patched manticore and they match with running Manticore EVM opcodes gas consumption. I narrowed down the reason for failures: self._get_memfee() returns incorrect gas to be consumed in some cases and in some cases SSTORE gas consumption looked suspicious. I also added xUnit XML report generation to the report and fixed gas mismatch with SELFDESTRUCT. I attach all xml test reports here, perhaps at some point it would be useful to hook everything into CI. |
Hi cryptometal. FYI. VMTests are going to be deprecated soon and ported to GeneralStateTests like this one: About the xml hooking it all in the CI. It sounds interesting. I'll need to investigate how to do that but you could file an issue about it if feel like it and we'll move the chat there. |
So pyevmasm supports multiple forks and mcore can selects how to use this for example in the unittests. We are still using VMTests to test mcores. Idea is to make it pass Frontier first using VMTest and then build up from there to newer forks (and GeneralStateTest)
|
Re-re-generate tests for the official vm tests from https://github.com/ethereum/tests/tree/develop/VMTests so it matches new mcore API.
Make concrete and symbolic version of each.
Document any tests that wont pass because of mcore design limitation
Make them pass
WIP here:
https://github.com/trailofbits/manticore/blob/dev-new-auto-evmtest/tests/auto/make_evmtests.py
Related:
#727
#1040
#1157
The text was updated successfully, but these errors were encountered: