Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error messages improvement #57

Closed
gumb0 opened this issue Nov 11, 2019 · 1 comment · Fixed by #70
Closed

Error messages improvement #57

gumb0 opened this issue Nov 11, 2019 · 1 comment · Fixed by #70

Comments

@gumb0
Copy link
Member

gumb0 commented Nov 11, 2019

  1. Client config file parsing error
> retesteth/retesteth -t GeneralStateTests -- --clients alethIPCDebug
Running tests using path: /home/andrei/dev/aleth/test/jsontests/
Running 51 test cases...
Active client configurations: 'alethIPCDebug '
unknown location(0): fatal error: in "GeneralStateTests/stCallCodes": dataobject::DataObjectException: expected ']' closing the array! around: stantinopleFix",
            "Istanbul"
}

/home/andrei/dev/retesteth/retesteth/testSuites/StateTestsBoost.cpp(81): last checkpoint: "stCallCodes" fixture ctor

Error message is confusing and should better mention that the error happened in the config file (now it looks like it's something about test file)

  1. Missing fork name in client config
> retesteth/retesteth -t GeneralStateTests -- --clients alethIPCDebug
Running tests using path: /home/andrei/dev/aleth/test/jsontests/
Running 51 test cases...
Active client configurations: 'alethIPCDebug '
Running tests for config 'Ethereum aleth on IPC Debug' 2
Test Case "stCallCodes": 
24%...
48%...
72%...
96%...
100%
Error: Specified network not found: 'Istanbul' (callcallcall_000_OOGMAfter)
Error: Specified network not found: 'Istanbul' (callcodecallcodecallcode_111_SuicideMiddle)
Error: Specified network not found: 'Istanbul' (callcallcallcode_001_SuicideMiddle)
Error: Specified network not found: 'Istanbul' (callcallcall_000_SuicideEnd)

Specified network not found - it's not clear where it is not found, config file is not mentioned

  1. Missing config for the fork
> retesteth/retesteth -t GeneralStateTests -- --clients alethIPCDebug
Running tests using path: /home/andrei/dev/aleth/test/jsontests/
Running 51 test cases...
Active client configurations: 'alethIPCDebug '
Finishing retesteth run
unknown location(0): fatal error: in "GeneralStateTests/stCallCodes": std::runtime_error: template .json config for network 'Istanbul' in alethIPCDebug

It's not clear at all that it looked for the config file and hasn't found it.

  1. Misssing additonalForks field in the client config file
> retesteth/retesteth -t GeneralStateTests/stRevertTest -- --clients alethIPCDebug
Running 1 test case...
Running tests using path: /home/andrei/dev/aleth/test/jsontests/
Active client configurations: 'alethIPCDebug '
Finishing retesteth run
unknown location(0): fatal error: in "GeneralStateTests/stRevertTest": std::runtime_error: additonalForks not found in ClientConfig  section! 
/home/andrei/dev/retesteth/retesteth/testSuites/StateTestsBoost.cpp(119): last checkpoint: "stRevertTest" fixture ctor

*** 1 failure is detected in the test module "Master Test Suite"

It's not really clear to the user, what is ClientConfig section and where they should fix it.

General tip: always include the path to the relevant file in error messages.

  1. -d/-v/-g filters out everything
> retesteth/retesteth -t GeneralStateTests/stPreCompiledContracts2 -- --clients alethIPCDebug --singletest CallRipemd160_1 -d 1
...
Error: Specified filter did not run a single transaction!  (stPreCompiledContracts2/CallRipemd160_1, fork: Istanbul, TrInfo: d: 0, g: 0, v: 0)

Maybe better There are no cases in the test for the given data/value/gas filter.
Should also say what did I requested (-d 1)

  1. Invalid test name
> retesteth/retesteth -t GeneralStateTests/stPreCompiledContracts2 -- --clients alethIPCDebug --singletest non-existing
Running 1 test case...
Running tests using path: /home/andrei/dev/aleth/test/jsontests/
Active client configurations: 'alethIPCDebug '
Filter: 'non-existing'
unknown location(0): fatal error: in "GeneralStateTests/stPreCompiledContracts2": test::BaseEthException: Could not find a filler for provided --singletest filter: 'non-existing', or no tests were found in the test suite folder!
/home/andrei/dev/retesteth/retesteth/testSuites/StateTestsBoost.cpp(88): last checkpoint: "stPreCompiledContracts2" fixture ctor

*** 1 failure is detected in the test module "Master Test Suite"

Don't confuse the user, please, and say exactly what's missing - a filler or a test.

@winsvega
Copy link
Collaborator

winsvega commented Nov 26, 2019

  1. Resolved by 1945410

  2. bcc7806

  3. 1fa84b5

  4. 7fdbe6c

  5. ac3f4cb

  6. 8bbe42b

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants