Unformatted “Windows Error 0x%X” exception message on Wine

[vadmium]

BPO	22977
Nosy	@vstinner, @tjguk, @vadmium, @zware, @serhiy-storchaka, @eryksun, @zooba
Files	win-error-format.patch win-error-format-v2.patch win-error-format-v3.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-04-06.17:40:25.386>
created_at = <Date 2014-12-02.11:19:44.468>
labels = ['easy', 'type-bug', 'OS-windows']
title = 'Unformatted \xe2\x80\x9cWindows Error 0x%X\xe2\x80\x9d exception message on Wine'
updated_at = <Date 2015-04-06.17:40:25.385>
user = 'https://github.com/vadmium'

bugs.python.org fields:

activity = <Date 2015-04-06.17:40:25.385>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2015-04-06.17:40:25.386>
closer = 'serhiy.storchaka'
components = ['Windows']
creation = <Date 2014-12-02.11:19:44.468>
creator = 'martin.panter'
dependencies = []
files = ['37678', '37686', '38641']
hgrepos = []
issue_num = 22977
keywords = ['patch', 'easy']
message_count = 21.0
messages = ['231987', '232046', '233873', '233876', '233899', '238978', '238986', '238989', '239875', '239877', '239878', '239882', '239885', '239888', '239891', '239892', '239910', '239949', '240164', '240166', '240168']
nosy_count = 9.0
nosy_names = ['vstinner', 'tim.golden', 'Arfrever', 'python-dev', 'martin.panter', 'zach.ware', 'serhiy.storchaka', 'eryksun', 'steve.dower']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue22977'
versions = ['Python 3.4', 'Python 3.5']

[vadmium]

The following code generates a connection reset error on Wine (Windows emulator, because I don’t have actual Windows to test on). Probably only a minor issue, but the error message isn’t quite right:

>>> s = create_connection(("localhost", 8181))
>>> # Server end accepts connection and then closes it
>>> s.sendall(b"3" * 3000000)                 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ConnectionResetError: [WinError 10054] Windows Error 0x%X

I’m just guessing, but looking at Python/errors.c, there are two PyUnicode_FromFormat("Windows Error 0x%X", ...) calls. The documentation for that function says only a lower-case %x is supported, so that would explain the behaviour I am seeing.

[eryksun]

This also affects SEH-related exceptions raised by ctypes. For example, VC++ uses exception code 0xE06D7363 (i.e. b'\xe0msc'), so unhandled VC++ exceptions leak into Python like this:

    >>> ctypes.windll.kernel32.RaiseException(0xe06d7363, 0, 0, None)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OSError: [WinError -529697949] Windows Error 0x%X

The ctypes SEH handler defaults to calling PyErr_SetFromWindowsErr(code). Since this isn't actually a Windows error code, Win32 FormatMessageW fails. Then Python uses the following default: PyUnicode_FromFormat("Windows Error 0x%X", err).

Normally (i.e. not under Wine) the OP's error number formats correctly:

    >>> ctypes.windll.kernel32.RaiseException(10054, 0, 0, None)     
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

PyErr_SetExcFromWindowsErrWithFilenameObjects
https://hg.python.org/cpython/file/ab2c023a9432/Python/errors.c#l553

PyErr_SetFromErrnoWithFilenameObjects
https://hg.python.org/cpython/file/ab2c023a9432/Python/errors.c#l416

[vadmium]

Here’s a simple patch which should fix it, although I have not verified this because I don’t have a Windows compiler (and MINGW cross compiling sounds too tricky)

[vstinner]

The attached patch lacks an unit test. When I will be able to build CPython again, I will try the patch.

[vadmium]

This patch includes a test case, based on Eryksun’s exception code

[vadmium]

V3 patch with suggested changes to the test case, though still completely untested by me.

[serhiy-storchaka]

LGTM.

[serhiy-storchaka]

There are other occurrences of %X in the code. Do you want provide a patch for them Martin?

[vadmium]

All the other occurrences of capitalized %X that I can find are not using Python’s string formatting functions. Please point them out if you can, but all I can see are some using a Microsoft vfwprintf_s() API, some calling standard C sprintf(), sscanf(), fprintf(), strftime() and strptime() APIs, and of course native Python 2-style string formatting, which I assume should all support %X.

[serhiy-storchaka]

Yes, you are right.

[python-dev]

New changeset 7907746baa0d by Serhiy Storchaka in branch '3.4':
Issue bpo-22977: Fixed formatting Windows error messages on Wine.
https://hg.python.org/cpython/rev/7907746baa0d

New changeset cf0cac11813d by Serhiy Storchaka in branch 'default':
Issue bpo-22977: Fixed formatting Windows error messages on Wine.
https://hg.python.org/cpython/rev/cf0cac11813d

[vstinner]

I'm not sure that Windows appreciate your change :-)

http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/6011/steps/test/logs/stdio

[216/393] test_exceptions
Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\PCbuild\..\lib\test\regrtest.py", line 1589, in <module>
    main_in_temp_cwd()
  File "C:\buildbot.python.org\3.x.kloth-win64\build\PCbuild\..\lib\test\regrtest.py", line 1564, in main_in_temp_cwd
    main()
  File "C:\buildbot.python.org\3.x.kloth-win64\build\PCbuild\..\lib\test\regrtest.py", line 738, in main
    raise Exception("Child error on {}: {}".format(test, result[1]))
Exception: Child error on test_exceptions: Exit code 3765269347

[vadmium]

I don’t pretend to know what is going on, or the best way to fix it. That exit code is the same code that my test passes to RaiseException. Perhaps it would be best to disable the test until someone with more knowledge or a Windows compiler can investigate.

[vstinner]

On Python 3.4, windll.kernel32.RaiseException(2, 0, 0, None) raised a FileNotFound error.

On Python 3.5, it displays a popup and the program exit.

It looks like the behaviour of RaiseException() changed in Python 3.5. I tested in debug and release mode.

@Steve: Any idea?

--

Instead of RaiseException, you can use ctypes.pythondll.PyErr_SetFromWindowsErr(code).

code = int.from_bytes(b"\xE0msc", "big")

Why not writing directly code = 3765269347?

The unit test should also check the exception message.

[vstinner]

Oh, and the ctypes module (import) must be optional, just skip the test if ctypes is missing.

http://buildbot.python.org/all/builders/AMD64%20OpenIndiana%203.x/builds/9555/steps/test/logs/stdio

test test_exceptions crashed -- Traceback (most recent call last):
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/test/regrtest.py", line 1267, in runtest_inner
    the_module = importlib.import_module(abstest)
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/importlib/__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2222, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2205, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2194, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1153, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1431, in exec_module
  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/test/test_exceptions.py", line 9, in <module>
    import ctypes
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/ctypes/__init__.py", line 7, in <module>
    from _ctypes import Union, Structure, Array
ImportError: No module named '_ctypes'

[serhiy-storchaka]

Could you provide a patch if you can test it Victor?

[python-dev]

New changeset dafae2b3c257 by Victor Stinner in branch '3.4':
Issue bpo-22977: Fix test_exceptions
https://hg.python.org/cpython/rev/dafae2b3c257

[vadmium]

Thanks for fixing the test Victor. The ctypes.pythonapi trick looks like a much better way. :)

[Arfrever]

Unconditional 'import ctypes' in Lib/test/test_exceptions.py was not yet deleted.

[python-dev]

New changeset 5cc0a090829a by Serhiy Storchaka in branch '3.4':
Issue bpo-22977: Remove unconditional import of ctypes.
https://hg.python.org/cpython/rev/5cc0a090829a

New changeset f46454229cf5 by Serhiy Storchaka in branch 'default':
Issue bpo-22977: Remove unconditional import of ctypes.
https://hg.python.org/cpython/rev/f46454229cf5

[serhiy-storchaka]

Thanks for your outsight Arfrever.