Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing d/v/g error vs missing test error #65

Closed
gumb0 opened this issue Nov 13, 2019 · 1 comment
Closed

Missing d/v/g error vs missing test error #65

gumb0 opened this issue Nov 13, 2019 · 1 comment

Comments

@gumb0
Copy link
Member

gumb0 commented Nov 13, 2019

I find it weird and confusing, that these two kinds of errors look quite different:

> retesteth/retesteth -t GeneralStateTests/stPreCompiledContracts2 -- --clients alethIPCDebug --singletest non-existing
Running 1 test case...
Running tests using path: /home/andrei/dev/aleth/test/jsontests/
Active client configurations: 'alethIPCDebug '
Filter: 'non-existing'
unknown location(0): fatal error: in "GeneralStateTests/stPreCompiledContracts2": test::BaseEthException: Could not find a filler for provided --singletest filter: 'non-existing', or no tests were found in the test suite folder!
/home/andrei/dev/retesteth/retesteth/testSuites/StateTestsBoost.cpp(88): last checkpoint: "stPreCompiledContracts2" fixture ctor

*** 1 failure is detected in the test module "Master Test Suite"
> retesteth/retesteth -t GeneralStateTests/stPreCompiledContracts2 -- --clients alethIPCDebug --singletest CallRipemd160_1 -d 1
Running 1 test case...
Running tests using path: /home/andrei/dev/aleth/test/jsontests/
Active client configurations: 'alethIPCDebug '
Filter: 'CallRipemd160_1 dInd: 1'
Running tests for config 'Ethereum aleth on IPC Debug' 2
Test Case "stPreCompiledContracts2": 
100%
Error: Specified filter did not run a single transaction!  (stPreCompiledContracts2/CallRipemd160_1, fork: Istanbul, TrInfo: d: 0, g: 0, v: 0)

--------
TestOutputHelper detected 1 errors during test execution!
/home/andrei/dev/retesteth/retesteth/TestOutputHelper.cpp(150): error: in "GeneralStateTests/stPreCompiledContracts2": 

*** 1 failure is detected in the test module "Master Test Suite"
Finishing retesteth run

--------
*** TOTAL ERRORS DETECTED: 1 errors during all test execution!
--------

It looks like the first one is a critical error and stops retesteth immediately, while with the second one retesteth continues to run till the final TOTAL ERRORS DETECTED message.

Is it because with -d 1 you can run many tests, so it continues to look for other ones? (is it even useful for anyone to use d/v/g filters with multiple tests at once?)

So I think it would be better to output the final "TOTAL ERRORS DETECTED" summary in the fist case, too.

@winsvega
Copy link
Collaborator

fixed with your previous issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants