Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Way to distinguish pytest tests when one function name is a prefix of another #142

Open
nwatson22 opened this issue Aug 25, 2023 · 2 comments
Labels
bug Something isn't working

Comments

@nwatson22
Copy link
Contributor

Pytest test selection using -k, which is what I have been using to isolate only one test for execution, doesn't work well when there are parentheses in the test name, e.g. "test_foundry_prove[AssertTest.test_assert_false()]", so to select this test I have to use something like poetry run pytest src/tests/integration --verbose -k "test_foundry_prove[AssertTest.test_assert_false". But this doesn't work well when there is another test function in the same contract, say, called test_assert_false_1(), since using this workaround "test_foundry_prove[AssertTest.test_assert_false" will also match this other test.

@nwatson22 nwatson22 added the bug Something isn't working label Aug 25, 2023
@ehildenb ehildenb transferred this issue from runtimeverification/evm-semantics Oct 27, 2023
@anvacaru
Copy link
Contributor

anvacaru commented Feb 2, 2024

This issue is still present.
If you run a test like

make test-integration TEST_ARGS+=' -k test_foundry_prove -k EnvTest.testEnvBool '

then pytest will run all the tests that match the provided name.

If you run it like

make test-integration TEST_ARGS+=' -k test_foundry_prove -k EnvTest.testEnvBool() '

then it complains about ( causing a syntax error

and if you add double quotes or escape the paranthesis like

make test-integration TEST_ARGS+=' -k test_foundry_prove -k "EnvTest.testEnvBool()" '

or

make test-integration TEST_ARGS+=' -k test_foundry_prove -k EnvTest.testEnvBool\(\) '

you get an internal error

INTERNALERROR> RuntimeError: Unexpectedly no active workers available

@anvacaru
Copy link
Contributor

anvacaru commented Feb 4, 2024

The INTERNALERROR> RuntimeError: Unexpectedly no active workers available goes away if numprocesseses is set to 0; But another error appears

poetry run pytest src/tests/integration --maxfail=1 --verbose --durations=0 --numprocesses=0 --dist=worksteal -k test_foundry_prove -k "EnvTest.testEnvBool()" 
================================ test session starts ================================
platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.4.0 -- /home/anvacaru/.cache/pypoetry/virtualenvs/kontrol-JS8qu14X-py3.10/bin/python
cachedir: .pytest_cache
rootdir: /home/anvacaru/rv/kontrol/master
plugins: cov-4.1.0, timeout-2.2.0, mock-3.12.0, pyk-0.1.611, xdist-3.5.0
collected 330 items                                                                 

=============================== no tests ran in 0.12s ===============================
ERROR: Wrong expression passed to '-k': EnvTest.testEnvBool(): at column 20: expected end of input; got left parenthesis

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants