pytest.mark.parametrize fails with lambdas

[mithrandi]

For example, the following:

import pytest
test = pytest.mark.parametrize('a', [1, 2, 3])(lambda a: None)

fails like this:

INTERNALERROR> Traceback (most recent call last):                                                                                                                                                                                     [1/950]
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/main.py", line 84, in wrap_session
INTERNALERROR>     doit(config, session)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/main.py", line 122, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 521, in __call__
INTERNALERROR>     return self._docall(self.methods, kwargs)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 528, in _docall
INTERNALERROR>     firstresult=self.firstresult).execute()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 394, in execute
INTERNALERROR>     res = method(*args)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/main.py", line 142, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 521, in __call__
INTERNALERROR>     return self._docall(self.methods, kwargs)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 528, in _docall
INTERNALERROR>     firstresult=self.firstresult).execute()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 393, in execute
INTERNALERROR>     return wrapped_call(method(*args), self.execute)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 113, in wrapped_call
INTERNALERROR>     return call_outcome.get_result()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 138, in get_result
INTERNALERROR>     py.builtin._reraise(*ex)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 123, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 394, in execute
INTERNALERROR>     res = method(*args)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/runner.py", line 65, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/runner.py", line 72, in runtestprotocol
INTERNALERROR>     rep = call_and_report(item, "setup", log)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/runner.py", line 121, in call_and_report
INTERNALERROR>     report = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 521, in __call__
INTERNALERROR>     return self._docall(self.methods, kwargs)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 528, in _docall
INTERNALERROR>     firstresult=self.firstresult).execute()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 393, in execute
INTERNALERROR>     return wrapped_call(method(*args), self.execute)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 109, in wrapped_call
INTERNALERROR>     wrap_controller.send(call_outcome)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/skipping.py", line 157, in pytest_runtest_makereport
INTERNALERROR>     rep = outcome.get_result()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 138, in get_result
INTERNALERROR>     py.builtin._reraise(*ex)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 123, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/core.py", line 394, in execute
INTERNALERROR>     res = method(*args)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/runner.py", line 227, in pytest_runtest_makereport
INTERNALERROR>     style=item.config.option.tbstyle)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/python.py", line 618, in _repr_failure_py
INTERNALERROR>     style=style)
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/main.py", line 392, in _repr_failure_py
INTERNALERROR>     return excinfo.value.formatrepr()
INTERNALERROR>   File "/home/mithrandi/deployment/virtualenvs/tempenv-1936316730d88/local/lib/python2.7/site-packages/_pytest/python.py", line 1540, in formatrepr
INTERNALERROR>     lines, _ = inspect.getsourcelines(function)
INTERNALERROR>   File "/usr/lib/python2.7/inspect.py", line 690, in getsourcelines
INTERNALERROR>     lines, lnum = findsource(object)
INTERNALERROR>   File "/usr/lib/python2.7/inspect.py", line 526, in findsource
INTERNALERROR>     file = getfile(object)
INTERNALERROR>   File "/usr/lib/python2.7/inspect.py", line 420, in getfile
INTERNALERROR>     'function, traceback, frame, or code object'.format(object))
INTERNALERROR> TypeError: <MarkDecorator 'parametrize' {'args': ('a', [1, 2, 3], <function <lambda> at 0x7fbd7d56aaa0>), 'kwargs': {}}> is not a module, class, method, function, traceback, frame, or code object

This reproducer may seem rather odd by itself, but something like this may arise with a decorator that returns a lambda.

I have confirmed this failure on pytest 2.7.0, 2.7.3, 2.8.0, and 2.8.1.

[nicoddemus]

Thanks for the report @mithrandi! 😄

[RonnyPfannschmidt]

this looks like a lambda is mistaken for a positional arggument

[Stranger6667]

Now the code above produces the following output:

cannot collect 'test' because it is not a function.

And it is not failure anymore. Tested with latest pytest version from repository on Python 2.7.12 & 3.5.1.

[RonnyPfannschmidt]

@mithrandi can thos one be closed based on the changes in master

[The-Compiler]

@RonnyPfannschmidt If something still fails but in a different way, I don't think it should be closed 😉

[RonnyPfannschmidt]

@The-Compiler as far as i can tell however we now have a normal report of the object there and the markdecorator invocation does consider a lambda an argument, not afunction that should be decorated

so i think with the change in reporting, everything works as designed

[nicoddemus]

I don't think we should support defining test functions using lambdas at all, I at least can't think of a valid use case for it. So I agree with @RonnyPfannschmidt that the error message is now appropriate, which I think was the heart of this issue.

@mithrandi what do you think?

[mithrandi]

The use case is that you are decorating a test function with pytest.mark.parametrize that was already decorated with another decorator that returns a lambda. As the user of these decorators, you don't really have control over the implementation of the decorators, and while you may be able to convince the implementor of the other decorator that they shouldn't use lambdas, the requirement seems a bit strange. Consider something simple like:

from functools import wraps
def enhance(f):
    return wraps(f)(lambda *a, **kw: f(enhancement, *a, **kw))

[RonnyPfannschmidt]

hmm, interesting, can you perhaps link to the library in question?

[mithrandi]

Unfortunately the exact circumstances where this came up are lost to me in the mists of time; I vaguely recall I was investigating something to do with https://github.com/HypothesisWorks/hypothesis-python but nothing beyond that.

[RonnyPfannschmidt]

i see, i propose closing then

[The-Compiler]

I can imagine there are other libaries doing this kind of thing. I'm going to be honest: I think pytest should support it, and it might be quite easy, but I'm not volunteering to do it - too much on my plate right now, sorry.

[nicoddemus]

In light of that, I agree with @The-Compiler that we should at least try to support it, if it's easy. What do you think @RonnyPfannschmidt? If so, I propose closing this one and opening another issue, like "Allow lambdas to be used as test functions" or so.

[RonnyPfannschmidt]

Nope, if the function name is <lambda>, the decorator is broken

Handling those different in marks means we are certain to break testsuites for bad reason

[nicoddemus]

Nope, if the function name is , the decorator is broken

I'm not sure that's the case here, at least with the example given by @mithrandi:

>>> from functools import wraps
>>> def enhance(f):
...     return wraps(f)(lambda *a, **kw: f(10, *a, **kw))
...
>>>
>>> @enhance
... def foo(v, x):
...     print v, x
...
>>> foo
<function foo at 0x0000000002AB55F8>
>>> foo(30)
10 30

Note that the name of the decorated function is foo, not lambda.

Is there anything special about lambdas that would prevent them from being collected as tests? I didn't take a look at the code, just wondering.

[RonnyPfannschmidt]

@nicoddemus the code in the marker specifically triggers for lambdas and ignores only them without checking into a wrapped function

[nicoddemus]

Oh I see, it complicates the marker code, not the collection code. Got it.

[The-Compiler]

Since we aren't sure if we should do this at all (I still think we should if possible...), this definitely shouldn't block 3.0.

[RonnyPfannschmidt]

closing as it seems we cant reproduce, please reopen if a reproducer happens to get known