Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parameterized ids can't be integers (was allowed in 2.9.2) #1857

Closed
okken opened this issue Aug 23, 2016 · 8 comments
Closed

Parameterized ids can't be integers (was allowed in 2.9.2) #1857

okken opened this issue Aug 23, 2016 · 8 comments
Labels

Comments

@okken
Copy link
Contributor

@okken okken commented Aug 23, 2016

Parameterized id's used to be able to be integers. Now that crashes verbosely.

  • [ x ] Include a detailed description of the bug

Used to be able to have numerical ids in paramterizations.
3.0.0 crashes verbosely if you do that.

  • [ x ] pip list of the virtual environment you are using
    (venv) $ pip list
    numpy (1.11.1)
    pip (8.1.2)
    py (1.4.31)
    pymongo (3.3.0)
    pytest (3.0.0)
    setuptools (20.10.1)
    unnecessary-math (0.0.1)
  • [ x ] pytest and operating system versions
    pytest 3.0.0 (tested against 2.9.2 and works there)
    os: mac something
  • [ x ] Minimal example if possible
import pytest

testdata = [( 1, 2), ( 2, 4)]

def times_2(x):
    return x * 2

@pytest.mark.parametrize("x,expected", testdata, ids=('a','b'))
def test_ids_strings(x,expected):
    '''works in both 3.0.0 and 2.9.2'''
    assert times_2(x) == expected


@pytest.mark.parametrize("x,expected", testdata, ids=(1,2))
def test_ids_numbers(x,expected):
    '''works in 2.9.2, crashes verbosely in 3.0.0'''
    assert times_2(x) == expected

2.9.2:

(venv_2.9.2) $ py.test -v test_ids2.py 
================================ test session starts =================================
platform darwin -- Python 3.5.2, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 -- /Users/okken/projects/book/bopytest/Book/code/pytest/um_project/tests/venv_2.9.2/bin/python3.5
cachedir: ../.cache
rootdir: /Users/okken/projects/book/bopytest/Book/code/pytest/um_project, inifile: 
collected 4 items 

test_ids2.py::test_ids_strings[a] PASSED
test_ids2.py::test_ids_strings[b] PASSED
test_ids2.py::test_ids_numbers[1] PASSED
test_ids2.py::test_ids_numbers[2] PASSED

============================== 4 passed in 0.02 seconds ==============================

3.0.0:

(venv) $ pytest -v test_ids2.py 
============================= test session starts ==============================
platform darwin -- Python 3.5.2, pytest-3.0.0, py-1.4.31, pluggy-0.3.1 -- /Users/okken/projects/book/bopytest/Book/venv/bin/python3.5
cachedir: ../.cache
rootdir: /Users/okken/projects/book/bopytest/Book/code/pytest/um_project, inifile: 
collected 0 items / 1 errors 

==================================== ERRORS ====================================
_____________________ ERROR collecting tests/test_ids2.py ______________________
../../../../venv/lib/python3.5/site-packages/_pytest/runner.py:163: in __init__
    self.result = func()
../../../../venv/lib/python3.5/site-packages/_pytest/main.py:460: in _memocollect
    return self._memoizedcall('_collected', lambda: list(self.collect()))
../../../../venv/lib/python3.5/site-packages/_pytest/main.py:331: in _memoizedcall
    res = function()
../../../../venv/lib/python3.5/site-packages/_pytest/main.py:460: in <lambda>
    return self._memoizedcall('_collected', lambda: list(self.collect()))
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:404: in collect
    return super(Module, self).collect()
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:318: in collect
    res = self.makeitem(name, obj)
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:330: in makeitem
    collector=self, name=name, obj=obj)
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:724: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:338: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:333: in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:595: in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:249: in _wrapped_call
    wrap_controller.send(call_outcome)
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:191: in pytest_pycollect_makeitem
    res = list(collector._genfunctions(name, obj))
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:350: in _genfunctions
    self.ihook.pytest_generate_tests(metafunc=metafunc)
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:724: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:338: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:333: in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
../../../../venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:596: in execute
    res = hook_impl.function(*args)
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:104: in pytest_generate_tests
    metafunc.parametrize(*marker.args, **marker.kwargs)
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:846: in parametrize
    ids = idmaker(argnames, argvalues, idfn, ids, self.config)
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:933: in idmaker
    for valindex, valset in enumerate(argvalues)]
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:933: in <listcomp>
    for valindex, valset in enumerate(argvalues)]
../../../../venv/lib/python3.5/site-packages/_pytest/python.py:929: in _idvalset
    return _escape_strings(ids[idx])
../../../../venv/lib/python3.5/site-packages/_pytest/compat.py:144: in _escape_strings
    return val.encode('unicode_escape').decode('ascii')
E   AttributeError: 'int' object has no attribute 'encode'
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.64 seconds ============================

@nicoddemus
Copy link
Member

@nicoddemus nicoddemus commented Aug 23, 2016

Hi @okken!

For the record, here's the output on 3.0.0:

=================================== ERRORS ====================================
_____________ ERROR collecting tmp/root/test_acceptance/test_0.py _____________
X:\pytest\_pytest\runner.py:163: in __init__
    self.result = func()
X:\pytest\_pytest\main.py:460: in _memocollect
    return self._memoizedcall('_collected', lambda: list(self.collect()))
X:\pytest\_pytest\main.py:331: in _memoizedcall
    res = function()
X:\pytest\_pytest\main.py:460: in <lambda>
    return self._memoizedcall('_collected', lambda: list(self.collect()))
X:\pytest\_pytest\python.py:405: in collect
    return super(Module, self).collect()
X:\pytest\_pytest\python.py:319: in collect
    res = self.makeitem(name, obj)
X:\pytest\_pytest\python.py:331: in makeitem
    collector=self, name=name, obj=obj)
X:\pytest\_pytest\vendored_packages\pluggy.py:724: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
X:\pytest\_pytest\vendored_packages\pluggy.py:338: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
X:\pytest\_pytest\vendored_packages\pluggy.py:333: in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
X:\pytest\_pytest\vendored_packages\pluggy.py:595: in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
X:\pytest\_pytest\vendored_packages\pluggy.py:249: in _wrapped_call
    wrap_controller.send(call_outcome)
X:\pytest\_pytest\python.py:192: in pytest_pycollect_makeitem
    res = list(collector._genfunctions(name, obj))
X:\pytest\_pytest\python.py:351: in _genfunctions
    self.ihook.pytest_generate_tests(metafunc=metafunc)
X:\pytest\_pytest\vendored_packages\pluggy.py:724: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
X:\pytest\_pytest\vendored_packages\pluggy.py:338: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
X:\pytest\_pytest\vendored_packages\pluggy.py:333: in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
X:\pytest\_pytest\vendored_packages\pluggy.py:596: in execute
    res = hook_impl.function(*args)
X:\pytest\_pytest\python.py:105: in pytest_generate_tests
    metafunc.parametrize(*marker.args, **marker.kwargs)
X:\pytest\_pytest\python.py:837: in parametrize
    ids = idmaker(argnames, argvalues, idfn, ids, self.config)
X:\pytest\_pytest\python.py:948: in idmaker
    for valindex, valset in enumerate(argvalues)]
X:\pytest\_pytest\python.py:948: in <listcomp>
    for valindex, valset in enumerate(argvalues)]
X:\pytest\_pytest\python.py:944: in _idvalset
    return _escape_strings(ids[idx])
X:\pytest\_pytest\compat.py:144: in _escape_strings
    return val.encode('unicode_escape').decode('ascii')
E   AttributeError: 'int' object has no attribute 'encode'
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.37 seconds ===========================

While I agree that the verbose error is far from ideal, I'm not sure we should fix it by supporting passing ints as ids. The docs for parametrize read:

    :arg ids: list of string ids, or a callable.
            If strings, each is corresponding to the argvalues so that they are
            part of the test id. If None is given as id of specific test, the
            automatically generated id for that argument will be used.
            If callable, it should take one argument (a single argvalue) and return
            a string or return None. If None, the automatically generated id for that
            argument will be used.
            If no ids are provided they will be generated automatically from
            the argvalues.

If we support passing ints (and probably any object which is not callable and we can call str on it), I'm afraid it will be more error prone because it will allow users to pass something they did not mean to and pytest will happily convert that to a string.

I think the proper fix here is to improve the error message, but still break the test suite with an error if ids are not a list of strings/None or a single callable. Fixing it is simple and I think it is better in the long run.

What do you think?

@okken
Copy link
Contributor Author

@okken okken commented Aug 23, 2016

An improved error message would suffice.

@okken
Copy link
Contributor Author

@okken okken commented Aug 23, 2016

I'm also noodling on what the "one value" is for "If callable, it should take one argument (a single argvalue) ...."
What is that one value? For fixtures and functions, the answer is different.

@nicoddemus
Copy link
Member

@nicoddemus nicoddemus commented Aug 23, 2016

I'll check, but I think it is either the name of the parameter or the fixture.

@nicoddemus
Copy link
Member

@nicoddemus nicoddemus commented Aug 23, 2016

What is that one value? For fixtures and functions, the answer is different.

Oh sorry, just realized you meant @parametrize on fixtures.

@nicoddemus
Copy link
Member

@nicoddemus nicoddemus commented Aug 24, 2016

Unfortunately the traceback is still there: #1860

@blueyed
Copy link
Contributor

@blueyed blueyed commented Nov 12, 2019

What about converting ints to strings?
This is useful with itertools.count() (#759).
Came here since https://github.com/blueyed/pytest/blob/44e071fe4c7b8d20a20dd82dc5de3fe29dcec400/testing/python/metafunc.py#L1188-L1204 fails, but I think it is good to support this.

blueyed added a commit to blueyed/pytest that referenced this issue Nov 12, 2019
@blueyed
Copy link
Contributor

@blueyed blueyed commented Nov 12, 2019

While I agree that the verbose error is far from ideal, I'm not sure we
should fix it by supporting passing ints as ids. The docs for parametrize
read:

    :arg ids: list of string ids, or a callable.
            If strings, each is corresponding to the argvalues so that they are
            part of the test id. If None is given as id of specific test, the
            automatically generated id for that argument will be used.
            If callable, it should take one argument (a single argvalue) and return
            a string or return None. If None, the automatically generated id for that
            argument will be used.
            If no ids are provided they will be generated automatically from
            the argvalues.

If we support passing ints (and probably any object which is not callable
and we can call str on it), I'm afraid it will be more error prone because
it will allow users to pass something they did not mean to and pytest will
happily convert that to a string.

We can just call str on ints only explicitly (blueyed@44e071f).
I think it is good to restore the old behavior.

blueyed added a commit to blueyed/pytest that referenced this issue Nov 12, 2019
blueyed added a commit to blueyed/pytest that referenced this issue Nov 12, 2019
Fixes pytest-dev#759

convert, tests

adjust test_parametrized_ids_invalid_type, create list to convert tuples

Ref: pytest-dev#1857 (comment)

changelog for int to str conversion

Ref: pytest-dev#1857 (comment)

coverage

_validate_ids: convert int/float/bool, doc revisit
blueyed added a commit to blueyed/pytest that referenced this issue Nov 13, 2019
Fixes pytest-dev#759

convert, tests

adjust test_parametrized_ids_invalid_type, create list to convert tuples

Ref: pytest-dev#1857 (comment)

changelog for int to str conversion

Ref: pytest-dev#1857 (comment)

coverage

_validate_ids: convert int/float/bool, doc revisit
blueyed added a commit to blueyed/pytest that referenced this issue Nov 13, 2019
Fixes pytest-dev#759

- Adjust test_parametrized_ids_invalid_type, create list to convert tuples
  Ref: pytest-dev#1857 (comment)

- Changelog for int to str conversion
  Ref: pytest-dev#1857 (comment)
blueyed added a commit to blueyed/pytest that referenced this issue Nov 13, 2019
blueyed added a commit to blueyed/pytest that referenced this issue Nov 13, 2019
blueyed added a commit to blueyed/pytest that referenced this issue Nov 14, 2019
blueyed added a commit to blueyed/pytest that referenced this issue Nov 18, 2019
Fixes pytest-dev#759

- Adjust test_parametrized_ids_invalid_type, create list to convert tuples
  Ref: pytest-dev#1857 (comment)

- Changelog for int to str conversion
  Ref: pytest-dev#1857 (comment)
blueyed added a commit to blueyed/pytest that referenced this issue Nov 19, 2019
Fixes pytest-dev#759

- Adjust test_parametrized_ids_invalid_type, create list to convert tuples
  Ref: pytest-dev#1857 (comment)

- Changelog for int to str conversion
  Ref: pytest-dev#1857 (comment)
blueyed added a commit to blueyed/pytest that referenced this issue Nov 19, 2019
Fixes pytest-dev#759

- Adjust test_parametrized_ids_invalid_type, create list to convert tuples
  Ref: pytest-dev#1857 (comment)

- Changelog for int to str conversion
  Ref: pytest-dev#1857 (comment)
blueyed added a commit to blueyed/pytest that referenced this issue Nov 20, 2019
Fixes pytest-dev#759

- Adjust test_parametrized_ids_invalid_type, create list to convert tuples
  Ref: pytest-dev#1857 (comment)

- Changelog for int to str conversion
  Ref: pytest-dev#1857 (comment)
blueyed added a commit to blueyed/pytest that referenced this issue Nov 20, 2019
Fixes pytest-dev#759

- Adjust test_parametrized_ids_invalid_type, create list to convert tuples
  Ref: pytest-dev#1857 (comment)

- Changelog for int to str conversion
  Ref: pytest-dev#1857 (comment)
blueyed added a commit to blueyed/pytest that referenced this issue Nov 21, 2019
Fixes pytest-dev#759

- Adjust test_parametrized_ids_invalid_type, create list to convert tuples
  Ref: pytest-dev#1857 (comment)

- Changelog for int to str conversion
  Ref: pytest-dev#1857 (comment)

(cherry picked from commit 2c941b5)

Conflicts:
	src/_pytest/mark/structures.py
	src/_pytest/python.py
	testing/python/metafunc.py
blueyed added a commit to blueyed/pytest that referenced this issue Nov 21, 2019
Fixes pytest-dev#759

- Adjust test_parametrized_ids_invalid_type, create list to convert tuples
  Ref: pytest-dev#1857 (comment)

- Changelog for int to str conversion
  Ref: pytest-dev#1857 (comment)

(cherry picked from commit 2c941b5)

Conflicts:
	src/_pytest/mark/structures.py
	src/_pytest/python.py
	testing/python/metafunc.py
blueyed added a commit to blueyed/pytest that referenced this issue Nov 21, 2019
Ref: pytest-dev#759

- Adjust test_parametrized_ids_invalid_type, create list to convert tuples
  Ref: pytest-dev#1857 (comment)

- Changelog for int to str conversion
  Ref: pytest-dev#1857 (comment)

(cherry picked from commit 2c941b5)

Conflicts:
	src/_pytest/mark/structures.py
	src/_pytest/python.py
	testing/python/metafunc.py
vinaycalastry added a commit to vinaycalastry/pytest that referenced this issue Dec 11, 2019
Fixes pytest-dev#759

- Adjust test_parametrized_ids_invalid_type, create list to convert tuples
  Ref: pytest-dev#1857 (comment)

- Changelog for int to str conversion
  Ref: pytest-dev#1857 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants