test_socket fails using Visual Studio 2010

[Kotan]

BPO	10469
Nosy	@loewis, @amauryfa, @pitrou, @kristjanvalur, @tjguk, @briancurtin
Superseder	bpo-13210: Support Visual Studio 2010
Files	test_socket.log: regrtest output pcbuild_vs2010.patch: patch for project and solution files for VS 2010 express edition issue10469.patch: switch preference of defines to prefer WSA* alternatives cpython_75849_to_75851.diff: WSA error patch from CCP

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-06-11.13:15:18.432>
created_at = <Date 2010-11-20.14:24:40.299>
labels = ['type-bug', 'library', 'build']
title = 'test_socket fails using Visual Studio 2010'
updated_at = <Date 2012-06-11.13:15:18.431>
user = 'https://bugs.python.org/Kotan'

bugs.python.org fields:

activity = <Date 2012-06-11.13:15:18.431>
actor = 'kristjan.jonsson'
assignee = 'none'
closed = True
closed_date = <Date 2012-06-11.13:15:18.432>
closer = 'kristjan.jonsson'
components = ['Build', 'Library (Lib)']
creation = <Date 2010-11-20.14:24:40.299>
creator = 'Kotan'
dependencies = []
files = ['19670', '19686', '19713', '24985']
hgrepos = []
issue_num = 10469
keywords = ['patch']
message_count = 18.0
messages = ['121651', '121652', '121659', '121688', '121692', '121738', '121741', '121748', '121753', '121762', '121767', '121770', '121784', '141149', '147889', '147891', '156495', '162614']
nosy_count = 9.0
nosy_names = ['loewis', 'amaury.forgeotdarc', 'pitrou', 'kristjan.jonsson', 'sable', 'tim.golden', 'brian.curtin', 'Kotan', 'Simon']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = '13210'
type = 'behavior'
url = 'https://bugs.python.org/issue10469'
versions = ['Python 3.1', 'Python 2.7', 'Python 3.2']

[Kotan]

Using WinXP I compiled python 3.2 from the current sources using Visual C++ 2010 Express.

Running

rt -v test_socket

resulted in the attached output.

The tests testSmallReadNonBlocking and testWriteNonBlocking have errors, and an assertion fails for test_connect and test_create_connections.

[Kotan]

current revision = svn revision 86553

[pitrou]

The core problem seems to be that the errno module has mismatching values on your Python build (for ECONNREFUSED and EWOULDBLOCK, at least, which explains all 4 failures). On a Windows 7 VM here:

>>> errno.ECONNREFUSED
10061
>>> errno.EWOULDBLOCK
10035

[Kotan]

As this was requested on IRC, I put the Visual C++ 2010 Express solution and project files here as well. The patch "pcbuild_vs2010.patch" should be applied to the PCbuild directory.

[briancurtin]

Daniel: If you need VS2008, you can get it here: http://www.microsoft.com/express/Downloads/bpo-2008-Visual-CPP

[loewis]

Kotan: please perform the following procedure:

1. Open pcbuild.sln
2. In the pythoncore project, open errnomodule.c
3. find the reference to EWOULDBLOCK and WSAEWOULDBLOCK
4. for each one, "go to definition"
5. report here what header files Visual Studio thinks they are defined in, and what values they have.

[Kotan]

#define EWOULDBLOCK     140
#define WSAEWOULDBLOCK                   10035L

The editor suggests that EWOULDBLOCK is already defined, therefore its code is used, i.e. 140.

#define ECONNREFUSED    107
#define WSAECONNREFUSED                  10061L

The E* are defined in errno.h and the WSA* in WinError.h. It seems that in my case the POSIX compliant headers are used.

[loewis]

OK, so what's the path to errno.h? In the errno.h included in my copy of VS 2010, the errno values only go up to 42 (80 if you count STRUNCATE), and EWOULDBLOCK is not defined in this header file.

[Kotan]

c:\Program Files\Microsoft Visual Studio 10.0\VC\include\errno.h

The content of my errno.h:

...
#define EILSEQ          42
#define STRUNCATE       80
#endif
#endif

/* Support EDEADLOCK for compatibility with older MS-C versions */
#define EDEADLOCK       EDEADLK

/* POSIX SUPPLEMENT */
#define EADDRINUSE      100
#define EADDRNOTAVAIL   101
...

[loewis]

Oops, I was confused. My copy is VS 2008, of course. So that's another incompatible change in VS 2010. Tcl, Ruby, and curl are ahead of us, here:

http://redmine.ruby-lang.org/issues/show/3092
http://sourceforge.net/tracker/index.php?func=detail&aid=3019634&group_id=10894&atid=310894
http://www.mail-archive.com/curl-library@cool.haxx.se/msg05164.html

I think we can work around that by switching the order of preference: use WSA constants if available, and errno.h values otherwise.

Kotan: can you please search the CRT sources whether these "POSIX" values are used anywhere?

[Kotan]

Only in the "system_error" file. (http://msdn.microsoft.com/en-us/library/ee372194.aspx) Seem to be a C++0X standard header file.

http://msdn.microsoft.com/en-us/library/5814770t.aspx

talks about the Posix codes only for compatibility reasons.

[loewis]

Only in the "system_error" file. (http://msdn.microsoft.com/en-us/library/ee372194.aspx)
Seem to be a C++0X standard header file.

(I assume you have actually scanned the crt\src folder as well)

Then it's indeed safe to ignore.

http://msdn.microsoft.com/en-us/library/5814770t.aspx

talks about the Posix codes only for compatibility reasons.

Unfortunately, they managed to break compatibility with that change,
instead of introducing it.

[Kotan]

Ok I did switched the preference for all "new" defines inside VC++ errno.h to use the WSA* alternatives if available.

Now test_socket passes for me, and the test_asyncore which did block before, now passes without a problem.

[simon]

Confirming this patch fixes the test_aynsc* tests in my VS10 build. Shouldn't it swap all WSA* defines to protect against this in the future, though? Alternatively, should the check for WSA* codes existing be in Lib\asyncore.py?

[sable]

I can also confirm that this patch corrects the problem with test_asyncore for Python 2.7 built with VS2010.

[briancurtin]

FYI: this would likely be handled through bpo-13210. I have a conversion sandbox started at http://hg.python.org/sandbox/vs2010port/ and am working through fixing test failures after the initial conversion.

[kristjanvalur]

Added a patch as used by CCP in production. Covers more WSA cases.

[kristjanvalur]

This has been fixed with the proper 2010 support