New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parameterized ids can't be integers (was allowed in 2.9.2) #1857

Closed
okken opened this Issue Aug 23, 2016 · 6 comments

Comments

Projects
None yet
2 participants
@okken
Contributor

okken commented Aug 23, 2016

Parameterized id's used to be able to be integers. Now that crashes verbosely.

  • [ x ] Include a detailed description of the bug

Used to be able to have numerical ids in paramterizations.
3.0.0 crashes verbosely if you do that.

  • [ x ] pip list of the virtual environment you are using
    (venv) $ pip list
    numpy (1.11.1)
    pip (8.1.2)
    py (1.4.31)
    pymongo (3.3.0)
    pytest (3.0.0)
    setuptools (20.10.1)
    unnecessary-math (0.0.1)
  • [ x ] pytest and operating system versions
    pytest 3.0.0 (tested against 2.9.2 and works there)
    os: mac something
  • [ x ] Minimal example if possible
import pytest

testdata = [( 1, 2), ( 2, 4)]

def times_2(x):
    return x * 2

@pytest.mark.parametrize("x,expected", testdata, ids=('a','b'))
def test_ids_strings(x,expected):
    '''works in both 3.0.0 and 2.9.2'''
    assert times_2(x) == expected


@pytest.mark.parametrize("x,expected", testdata, ids=(1,2))
def test_ids_numbers(x,expected):
    '''works in 2.9.2, crashes verbosely in 3.0.0'''
    assert times_2(x) == expected

2.9.2:

(venv_2.9.2) $ py.test -v test_ids2.py 
================================ test session starts =================================
platform darwin -- Python 3.5.2, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 -- /Users/okken/projects/book/bopytest/Book/code/pytest/um_project/tests/venv_2.9.2/bin/python3.5
cachedir: ../.cache
rootdir: /Users/okken/projects/book/bopytest/Book/code/pytest/um_project, inifile: 
collected 4 items 

test_ids2.py::test_ids_strings[a] PASSED
test_ids2.py::test_ids_strings[b] PASSED
test_ids2.py::test_ids_numbers[1] PASSED
test_ids2.py::test_ids_numbers[2] PASSED

============================== 4 passed in 0.02 seconds ==============================

3.0.0:

(venv) $ pytest -v test_ids2.py 
============================= test session starts ==============================
platform darwin -- Python 3.5.2, pytest-3.0.0, py-1.4.31, pluggy-0.3.1 -- /Users/okken/projects/book/bopytest/Book/venv/bin/python3.5
cachedir: ../.cache
rootdir: /Users/okken/projects/book/bopytest/Book/code/pytest/um_project, inifile: 
collected 0 items / 1 errors 

==================================== ERRORS ====================================
_____________________ ERROR collecting tests/test_ids2.py ______________________
../../../../venv/lib/python3.5/site-packages/_pytest/runner.py:163: in __init__
    self.result = func()
../../../../venv/lib/python3.5/site-packages/_pytest/main.py:460: in _memocollect
    return self._memoizedcall('_collected', lambda: list(self.collect()))
../../../../venv/lib/python3.5/site-packages/_pytest/main.py:331: in _memoizedcall
    res = function()
../../../../venv/lib/python3.5/site-packages/_pytest/main.py:460: in <lambda>
    return self._memoizedcall('_collected', lambda: list(self.collect()))
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:404: in collect
    return super(Module, self).collect()
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:318: in collect
    res = self.makeitem(name, obj)
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:330: in makeitem
    collector=self, name=name, obj=obj)
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:724: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:338: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:333: in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:595: in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:249: in _wrapped_call
    wrap_controller.send(call_outcome)
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:191: in pytest_pycollect_makeitem
    res = list(collector._genfunctions(name, obj))
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:350: in _genfunctions
    self.ihook.pytest_generate_tests(metafunc=metafunc)
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:724: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:338: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:333: in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:596: in execute
    res = hook_impl.function(*args)
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:104: in pytest_generate_tests
    metafunc.parametrize(*marker.args, **marker.kwargs)
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:846: in parametrize
    ids = idmaker(argnames, argvalues, idfn, ids, self.config)
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:933: in idmaker
    for valindex, valset in enumerate(argvalues)]
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:933: in <listcomp>
    for valindex, valset in enumerate(argvalues)]
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:929: in _idvalset
    return _escape_strings(ids[idx])
../../../../venv/lib/python3.5/site-packages/_pytest/compat.py:144: in _escape_strings
    return val.encode('unicode_escape').decode('ascii')
E   AttributeError: 'int' object has no attribute 'encode'
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.64 seconds ============================

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Aug 23, 2016

Hi @okken!

For the record, here's the output on 3.0.0:

=================================== ERRORS ====================================
_____________ ERROR collecting tmp/root/test_acceptance/test_0.py _____________
X:\pytest\_pytest\runner.py:163: in __init__
    self.result = func()
X:\pytest\_pytest\main.py:460: in _memocollect
    return self._memoizedcall('_collected', lambda: list(self.collect()))
X:\pytest\_pytest\main.py:331: in _memoizedcall
    res = function()
X:\pytest\_pytest\main.py:460: in <lambda>
    return self._memoizedcall('_collected', lambda: list(self.collect()))
X:\pytest\_pytest\python.py:405: in collect
    return super(Module, self).collect()
X:\pytest\_pytest\python.py:319: in collect
    res = self.makeitem(name, obj)
X:\pytest\_pytest\python.py:331: in makeitem
    collector=self, name=name, obj=obj)
X:\pytest\_pytest\vendored_packages\pluggy.py:724: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
X:\pytest\_pytest\vendored_packages\pluggy.py:338: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
X:\pytest\_pytest\vendored_packages\pluggy.py:333: in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
X:\pytest\_pytest\vendored_packages\pluggy.py:595: in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
X:\pytest\_pytest\vendored_packages\pluggy.py:249: in _wrapped_call
    wrap_controller.send(call_outcome)
X:\pytest\_pytest\python.py:192: in pytest_pycollect_makeitem
    res = list(collector._genfunctions(name, obj))
X:\pytest\_pytest\python.py:351: in _genfunctions
    self.ihook.pytest_generate_tests(metafunc=metafunc)
X:\pytest\_pytest\vendored_packages\pluggy.py:724: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
X:\pytest\_pytest\vendored_packages\pluggy.py:338: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
X:\pytest\_pytest\vendored_packages\pluggy.py:333: in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
X:\pytest\_pytest\vendored_packages\pluggy.py:596: in execute
    res = hook_impl.function(*args)
X:\pytest\_pytest\python.py:105: in pytest_generate_tests
    metafunc.parametrize(*marker.args, **marker.kwargs)
X:\pytest\_pytest\python.py:837: in parametrize
    ids = idmaker(argnames, argvalues, idfn, ids, self.config)
X:\pytest\_pytest\python.py:948: in idmaker
    for valindex, valset in enumerate(argvalues)]
X:\pytest\_pytest\python.py:948: in <listcomp>
    for valindex, valset in enumerate(argvalues)]
X:\pytest\_pytest\python.py:944: in _idvalset
    return _escape_strings(ids[idx])
X:\pytest\_pytest\compat.py:144: in _escape_strings
    return val.encode('unicode_escape').decode('ascii')
E   AttributeError: 'int' object has no attribute 'encode'
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.37 seconds ===========================

While I agree that the verbose error is far from ideal, I'm not sure we should fix it by supporting passing ints as ids. The docs for parametrize read:

    :arg ids: list of string ids, or a callable.
            If strings, each is corresponding to the argvalues so that they are
            part of the test id. If None is given as id of specific test, the
            automatically generated id for that argument will be used.
            If callable, it should take one argument (a single argvalue) and return
            a string or return None. If None, the automatically generated id for that
            argument will be used.
            If no ids are provided they will be generated automatically from
            the argvalues.

If we support passing ints (and probably any object which is not callable and we can call str on it), I'm afraid it will be more error prone because it will allow users to pass something they did not mean to and pytest will happily convert that to a string.

I think the proper fix here is to improve the error message, but still break the test suite with an error if ids are not a list of strings/None or a single callable. Fixing it is simple and I think it is better in the long run.

What do you think?

@okken

This comment has been minimized.

Contributor

okken commented Aug 23, 2016

An improved error message would suffice.

@okken

This comment has been minimized.

Contributor

okken commented Aug 23, 2016

I'm also noodling on what the "one value" is for "If callable, it should take one argument (a single argvalue) ...."
What is that one value? For fixtures and functions, the answer is different.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Aug 23, 2016

I'll check, but I think it is either the name of the parameter or the fixture.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Aug 23, 2016

What is that one value? For fixtures and functions, the answer is different.

Oh sorry, just realized you meant @parametrize on fixtures.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Aug 24, 2016

Unfortunately the traceback is still there: #1860

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment