Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keyword selection expressions are limited to python syntax, which excludes some nodeids #1141

Closed
jaysw opened this issue Oct 16, 2015 · 5 comments · Fixed by #7122
Closed
Labels
status: help wanted developers would like help from experts on this topic topic: selection related to test selection from the command line type: backward compatibility might present some backward compatibility issues which should be carefully noted in the changelog type: proposal proposal for a new feature, often to gather opinions or design the API around the new feature

Comments

@jaysw
Copy link

jaysw commented Oct 16, 2015

Hi there.

py.test is fantastic!

I'm wondering if the following is an error, or, expected behaviour:

With the following:

@pytest.mark.parametrize('p', ['spam eggs', 'ham'])
def test_me(p):
    assert 1 == 1

This works fine:

py.test -k 'test_me[ham]'

While the following triggers an internal error / syntax error:

py.test -k 'test_me[spam eggs]'

Looks like condition here:

https://github.com/pytest-dev/pytest/blob/master/_pytest/mark.py#L154

prevents any matching on parameters having spaces in them (and falls through to an eval() call).

@RonnyPfannschmidt
Copy link
Member

The current mark keyword evaluator uses python syntax,
There is a plan to create a more free evaluator, but its much work and not a priority

@RonnyPfannschmidt RonnyPfannschmidt added type: bug problem that needs to be addressed status: help wanted developers would like help from experts on this topic topic: collection related to the collection phase type: backward compatibility might present some backward compatibility issues which should be carefully noted in the changelog labels Oct 17, 2015
@jaysw
Copy link
Author

jaysw commented Oct 19, 2015

So, in the above example, is there a way to select and run just the test, "test_me[spam eggs]" ?

@RonnyPfannschmidt
Copy link
Member

The exact test can be selected by nodeid just use py.test $nodeid

@RonnyPfannschmidt RonnyPfannschmidt changed the title INTERNALERROR> SyntaxError, while collecting tests. Mark expressions are limited to python syntax which is only a subset of what can be a nodeid Oct 25, 2015
@Zac-HD Zac-HD added type: proposal proposal for a new feature, often to gather opinions or design the API around the new feature topic: selection related to test selection from the command line and removed topic: collection related to the collection phase type: bug problem that needs to be addressed labels Oct 20, 2018
@Zac-HD Zac-HD changed the title Mark expressions are limited to python syntax which is only a subset of what can be a nodeid Keyword selection expressions are limited to python syntax, which excludes some nodeids Oct 21, 2018
@Zac-HD
Copy link
Member

Zac-HD commented Oct 21, 2018

To summarise, as of pytest 3.9.1:

pytest 'test.py::test_me[spam eggs]'    # Selection by nodeid works
pytest test.py -k 'test_me[spam eggs]'  # UsageError in _pytest/mark/legacy.py

The root cause is this code (updated link), which chokes when trying to eval a string which is not valid Python. The only obvious solution is to upgrade that function, as using the repr rather than the str in nodeids would have severe compatibility issues. Unclear if we want to bother though, since it's in a legacy module that may be removed soon?

@RonnyPfannschmidt
Copy link
Member

We don't have a replacement lined up yet

bluetech added a commit to bluetech/pytest that referenced this issue Apr 25, 2020
Previously, the expressions given to the `-m` and `-k` options were
evaluated with `eval`. This causes a few issues:

- Python keywords cannot be used.

- Constants like numbers, None, True, False are not handled correctly.

- Various syntax like numeric operators and `X if Y else Z` is supported
  unintentionally.

- `eval()` is somewhat dangerous for arbitrary input.

- Can fail in many ways so requires `except Exception`.

The format we want to support is quite simple, so change to a custom
parser. This fixes the issues above, and gives us full control of the
format, so can be documented comprehensively and even be extended in the
future if we wish.

Fixes pytest-dev#1141.
Fixes pytest-dev#3573.
Fixes pytest-dev#5881.
Fixes pytest-dev#6822.
Fixes pytest-dev#7112.
bluetech added a commit to bluetech/pytest that referenced this issue Apr 25, 2020
Previously, the expressions given to the `-m` and `-k` options were
evaluated with `eval`. This causes a few issues:

- Python keywords cannot be used.

- Constants like numbers, None, True, False are not handled correctly.

- Various syntax like numeric operators and `X if Y else Z` is supported
  unintentionally.

- `eval()` is somewhat dangerous for arbitrary input.

- Can fail in many ways so requires `except Exception`.

The format we want to support is quite simple, so change to a custom
parser. This fixes the issues above, and gives us full control of the
format, so can be documented comprehensively and even be extended in the
future if we wish.

Fixes pytest-dev#1141.
Fixes pytest-dev#3573.
Fixes pytest-dev#5881.
Fixes pytest-dev#6822.
Fixes pytest-dev#7112.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: help wanted developers would like help from experts on this topic topic: selection related to test selection from the command line type: backward compatibility might present some backward compatibility issues which should be carefully noted in the changelog type: proposal proposal for a new feature, often to gather opinions or design the API around the new feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants