test_invalid_utf8_arg fails in non-UTF-8 multibyte locales

[serhiy-storchaka]

Bug report

test.test_cmd_line.CmdLineTest.test_invalid_utf8_arg fails in a non-UTF-8 multibyte locale such as ja_JP.eucjp:

$ LC_ALL=ja_JP.eucjp ./python -m test test_cmd_line -m test_invalid_utf8_arg
...
AssertionError: '' != "b'\\xff\\xc3\\xff...'"

The test passes arbitrary bytes as argv and checks they round-trip via os.fsencode(sys.argv[1]). But that round-trip is only lossless for UTF-8 and single-byte encodings; in a stateful multibyte encoding like EUC-JP it isn't (and Py_DecodeLocale()'s C decoder diverges from the Python codec anyway), so the run_default and run_no_utf8_mode variants — which use the filesystem / locale encoding — fail.

Linked PRs

• gh-154582: Fix test_invalid_utf8_arg in non-UTF-8 multibyte locales #154584
• [3.15] gh-154582: Fix test_invalid_utf8_arg in non-UTF-8 multibyte locales (GH-154584) #154601
• [3.14] gh-154582: Fix test_invalid_utf8_arg in non-UTF-8 multibyte locales (GH-154584) #154604
• [3.13] gh-154582: Fix test_invalid_utf8_arg in non-UTF-8 multibyte locales (GH-154584) #154605