pytest -k should ignore private attributes (starting with _)

[asmeurer]

I originally opened this issue in hypothesis, but they told me to open it here HypothesisWorks/hypothesis#2508.

When I run pytest -k name, and name only matches hypothesis tests, pytest runs all hypothesis tests, not just the ones that match. When name matches a non-hypothesis test, it only picks the tests that contain name as expected.

I am using pytest 5.4.3 and hypothesis 5.21.0. It happens even when I set aside my pytest.ini and conftest.py.

You can reproduce this at the repo https://github.com/Quansight/ndindex/ by running pytest -v -k repr.

[asottile]

hypothesis is attaching additional attributes to the test function which is triggering this:

{'is_hypothesis_test', '_hypothesis_internal_use_settings', 'test_ellipsis.py', 'hypothesis', '_hypothesis_internal_use_reproduce_failure', '_hypothesis_internal_use_seed', 'tests', 'test_ellipsis_hypothesis'}

repr is matching _hypothesis_internal_use_reproduce_failure

[asmeurer]

Is there a way to make -k only match function names? I don't care about that other stuff.

[nicoddemus]

Hi @asmeurer, I'm afraid not at the moment with the -k flag, it always matches partially against node names and "keywords".

[asottile]

this behaviour has existed since pytest 2.4.0 but as far as I can tell is undocumented (and wasn't called out in the changelog)

I think at least this should be documented and improved to avoid underscored names (this would solve this particular issue, but I think would be a reasonable baseline expectation)

[nicoddemus]

I think at least this should be documented

Created #7546 to track this.