2731.bug Fix ReprFuncArgs with mixed unicode and utf-8 args.

[fgmacedo]

The class _pytest._code.code.ReprFuncArgs crashes with UnicodeDecodeError when trying to write args with mixed unicode and utf-8 strings.

Fixes #2731.

[coveralls]

Coverage remained the same at 91.805% when pulling 4d6d450 on fgmacedo:fgm-fix-toterminal into 709b8b6 on pytest-dev:master.

[nicoddemus]

Thanks @fgmacedo!

Could you please also include a regression test similar to what you posted on the issue and a changelog entry?

[nicoddemus]

For the changelog entry, I suggest a file changelog/2731.bug with the contents:

Fix ``ReprFuncArgs`` with mixed unicode and UTF-8 args.

[fgmacedo]

Yes, for sure! Any hint for where to put the regression test? (I'm reading the test_terminal.py module).

[fgmacedo]

Hi @nicoddemus, I think that we have reached another issue:
I've wrote the following test:

    def test_setup_fixture_error_with_mixed_encoding(self, testdir):
        testdir.makepyfile("""
            def mixed_encoding(as_unicode, as_utf8):
                assert 0
            def setup_function(function):
                mixed_encoding(u"'São Paulo'", "'S\xc3\xa3o Paulo'")
            def test_nada():
                pass
        """)
        result = testdir.runpytest()
        result.stdout.fnmatch_lines([
            "*ERROR at setup of test_nada*",
            "*setup_function(function):*",
            "*as_unicode = \"'São Paulo'\", as_utf8 = \"'São Paulo'\"*",
            "*mixed_encoding(as_unicode, as_utf8):*",
            "*assert 0*",
            "*1 error*",
        ])
        assert result.ret != 0

It works fine on Python 3.6, but fails on Python 2.7:

============================================================================= FAILURES =============================================================================
________________________________________________ TestFixtureReporting.test_setup_fixture_error_with_mixed_encoding _________________________________________________

self = <test_terminal.TestFixtureReporting object at 0x7f2d69b03f90>
testdir = <Testdir local('/tmp/pytest-of-fernando/pytest-33/testdir/test_setup_fixture_error_with_mixed_encoding0')>

    def test_setup_fixture_error_with_mixed_encoding(self, testdir):
        testdir.makepyfile("""
                def mixed_encoding(as_unicode, as_utf8):
                    assert 0
                def setup_function(function):
                    mixed_encoding(u"'São Paulo'", "'S\xc3\xa3o Paulo'")
                def test_nada():
                    pass
>           """)

/opt/loggi/pytest/testing/test_terminal.py:363: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/loggi/pytest/.tox/py27/lib/python2.7/site-packages/_pytest/pytester.py:550: in makepyfile
    return self._makefile('.py', args, kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Testdir local('/tmp/pytest-of-fernando/pytest-33/testdir/test_setup_fixture_error_with_mixed_encoding0')>, ext = '.py'
args = ('\n            def mixed_encoding(as_unicode, as_utf8):\n                assert 0\n            def setup_function(fun...oding(u"\'S\xc3\xa3o Paulo\'", "\'S\xc3\xa3o Paulo\'")\n            def test_nada():\n                pass\n        ',)
kwargs = {}, encoding = 'utf-8'

    def _makefile(self, ext, args, kwargs, encoding="utf-8"):
        items = list(kwargs.items())
        if args:
            source = py.builtin._totext("\n").join(
>               map(py.builtin._totext, args)) + py.builtin._totext("\n")
E           UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 156: ordinal not in range(128)

/opt/loggi/pytest/.tox/py27/lib/python2.7/site-packages/_pytest/pytester.py:493: UnicodeDecodeError
================================================= 1 failed, 1841 passed, 61 skipped, 13 xfailed in 187.53 seconds ==================================================
ERROR: InvocationError: '/opt/loggi/pytest/.tox/py27/bin/pytest --lsof -rfsxX testing'

[coveralls]

Coverage remained the same at 91.805% when pulling 09c314b on fgmacedo:fgm-fix-toterminal into 709b8b6 on pytest-dev:master.

[fgmacedo]

Closing this in favor of #2735

[coveralls]

Coverage remained the same at 91.805% when pulling bf854a9 on fgmacedo:fgm-fix-toterminal into 709b8b6 on pytest-dev:master.