Is there anyway to not escape id in parametrization?

[youknowone]

Thanks for submitting an issue!

Here's a quick checklist in what to include:

• Include a detailed description of the bug or suggestion
• pip list of the virtual environment you are using
• pytest and operating system versions
• Minimal example if possible

---

I want to print original unicode text without escaping, but I can't find anyway to do that.

code:

import pytest

@pytest.mark.parametrize('text,result', [
    ['테스트1', False],
    ['테스트2', False],
])
def test_func(text):
    assert False

result of pytest run

$ pytest t.py
======= test session starts =========
platform darwin -- Python 3.6.1, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 -- ~/.pyenv/versions/3.6.1/bin/python
cachedir: .cache
plugins: asyncio-0.5.0
collected 2 items

t.py::test_func[\ud14c\uc2a4\ud2b81-False] FAILED
t.py::test_func[\ud14c\uc2a4\ud2b82-False] FAILED

Expected result:

t.py::test_func[테스트1-False] FAILED
t.py::test_func[테스트2-False] FAILED

Unfortunately, I found you are regarding this as a feature in: #1470

So, for now, I am doing that by adding:

import _pytest
_pytest.python._escape_strings = lambda x: x

I really need to see the original unicode text, because escaped characters are not human readable. At least, it is a lot better to be displayed in both escaped and unescaped way, not just in escaped way.

Is there any better way?

[The-Compiler]

So in an ideal world, we'd be able to use unicode everywhere, without anything breaking. Unfortunately, that's not how things are...

Those test IDs are used in various places - on Windows, even printing them if they're non-ascii can be problematic, and we also potentially use them as filenames and in other problematic places. So them being non-ascii isn't really an option I think, because it just breaks too much stuff 😟

The best thing you can do is defining an explicit ID where needed with an appropriate transliteration to ascii, for example (if Google Translate doesn't betray me):

import pytest

@pytest.mark.parametrize('text', [
    pytest.param('테스트', id='teseuteu'),
])
def test_func(text):
    assert False

[youknowone]

Thanks for the explanation. I understand your concerns.

But in my case, I have tons of that kind of parametrization everywhere. Adding code I am using is a lot easier than adding id everywhere. Also, Because the project never run in windows, I still want to print unicode in terminal.

My suggestion: How about adding an option like escape_parametrization_id = False in pytest.ini or setup.cfg?

Believe or not, it is related to quality of life who works for non-English environments and they will take the risk of breaking it on their own responsibility.

[nicoddemus]

How about adding an option like escape_parametrization_id = False in pytest.ini or setup.cfg?

Seems reasonable to me. Would you like to contribute a PR with that? This could go into pytest-3.2.0 then.

[The-Compiler]

Sounds like a good idea to me too, as long as the doc explains the problems with it.

I do definitely believe "Believe or not, it is related to quality of life who works for non-English environments and they will take the risk of breaking it on their own responsibility." though, I wouldn't want to have to use a foreign script for my test IDs either! 😉

also cc'ing @ceridwen @RonnyPfannschmidt @davehunt who were involved with related changes.

[RonnyPfannschmidt]

each option has quite a cost, - in particular when it comes to unicode

this would create quite a space for strange errors and i would strongly advice against it - personally i'd say NO to such a feature as i cant even beginn to understand the kinds of problems enabling it and testing it opens up

wit the current structure of pytest we cant possibly hope to provide a feature such as this one in a consistent and well tested manner

[nicoddemus]

wit the current structure of pytest we cant possibly hope to provide a feature such as this one in a consistent and well tested manner

I think the purpose here would provide a means for users to use unicode in their test ids, but without any guarantee that it would work everywhere. I agree with @The-Compiler when he says as long as the doc explains the problems with it.

I would go so far as adding this option inside a Note in the docs explaining the possible problems and that users can enable this at their own risk.

[ApaDoctor]

I'm interested in help out.
What’s the best way to start?

[nicoddemus]

For starters it would need to add an ini option for it and use that option in python.py to not call _escape_strings if that option is set.

I would document it inside a note in the parametrization docs like this:

Note: pytest by default escapes any non-ascii characters used in unicode strings for the parametrization because it has several downsides like (...). If however you would like to use them, use the option escape_parametrization_id = False to disable this behavior, but keep in mind that this might cause unwanted side effects and even bugs depending on the OS used and plugins currently installed, so use it at your own risk.

But before you start anything, I would still ask @RonnyPfannschmidt what he thinks about the general idea.

[youknowone]

Thanks everyone.

I found it is a little bit more complicated. _escape_strings not only escape unicode but also process other stuffs like newline. So it seems the appropriate way is adding parameter to _escape_strings or reading config inside of _escape_strings.
(or maybe the new line processing is just a part of general escaping)

[ApaDoctor]

@RonnyPfannschmidt, so what do you think?

[RonnyPfannschmidt]

i still doubt its a good idea - the option name should be something really painful

[nicoddemus]

the option name should be something really painful

You mean to implement it?

[RonnyPfannschmidt]

yes - enabling it should make absolutely clear that the end result wi lbe unsupported, and we wont fix any errors/problems based on it

something like disble_test_id_escaping_and_forfeit_all_rights_to_community_support = true

it should put a warning into the test header as well

if we cant commit to it having a horrible name like that, we should not implement it at all

[dimaqq]

I feel this feature deserves a plugin :)

It could also be mainstream, as Python is all about unicode-everywhere in the latest releases, but then, would it have to be limited to py3.7+?

Ref: https://vstinner.github.io/python37-new-utf8-mode.html

[youknowone]

Thanks @ApaDoctor