New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mention "-k not" option on the documentation #1442

Closed
keszybz opened this Issue Mar 8, 2016 · 7 comments

Comments

Projects
None yet
5 participants
@keszybz

keszybz commented Mar 8, 2016

I'm sure this must have been discussed before, but I cannot find it in the issue list or documentation...

A way to skip tests from the command-line would be really nice. Currently it's possible to

  • skip whole files with --ignore=path
  • add skipif markers in the code
    but there doesn't seem to be the equivalent of nosetest's --exclude=REGEX option that matches on test names.

(This is for example very useful when packaging stuff, it's much easier to skip certain known flaky or broken tests from the commandline when invoking py.test, than to modify the sources.)

@The-Compiler

This comment has been minimized.

Member

The-Compiler commented Mar 9, 2016

You can use -k (keyword selection) with not to do so: py.test -k 'not foo' will deselect all tests containing foo.

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Mar 9, 2016

we indeed to not have a way to ignore specific parts of the collection tree

@keszybz

This comment has been minimized.

keszybz commented Mar 9, 2016

Thanks, not works indeed. So maybe it's enough to update the docs?

  -k EXPRESSION         only run tests which match the given substring
                        expression. An expression is a python evaluatable
                        expression where all names are substring-matched
                        against test names and their parent classes. Example:
                        -k 'test_method or test other' matches all test
                        functions and classes whose name contains
                        'test_method' or 'test_other'. Additionally keywords
                        are matched to classes and functions containing extra
                        names in their 'extra_keyword_matches' set, as well as
                        functions which have names assigned directly to them.

Mentioning not would be really useful here.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Mar 9, 2016

(Updated the issue title, hope you don't mind)

@keszybz would you care to open a PR with that? 😁

@nicoddemus nicoddemus changed the title from allow excluding tests from the command line to Mention "-k not" option on the documentation Mar 9, 2016

@keszybz

This comment has been minimized.

keszybz commented Mar 9, 2016

Sure, I can try to update the docs, but I want to nail down all the details first.

I have a project with the same test name in two different files. I was hoping

py.test-3 -vv -k 'not test_fsimage.py::test_differences'

would DTRT, but unfortunately it doesn't work. Is it possible to do something like that (match on both filename and test name)?

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Mar 11, 2016

Hmm I'm not sure, I think -k works only for "keywords", so for test_fsimage.py::test_differences I think pytest will have this keywords to match: ['test_fsimage.py', 'test_differences']

@xuanluong

This comment has been minimized.

Contributor

xuanluong commented Sep 30, 2017

@nicoddemus I think the usage of -k not is already mentioned in the documentation

This will run tests which contain names that match the given string expression, which can include Python operators that use filenames, class names and function names as variables

As an operator, not can be implied to be usable in an expression.

So what's really lacking in the documentation here is mentioning the availability of -k not in the help of -k option: https://github.com/pytest-dev/pytest/blob/master/_pytest/mark.py#L89

I can update the help text a bit to make it clearer that not and some other Python operators are available. Can I take over this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment