test_socket failure on AIX

[DavidEdelsohn]

BPO	22397
Nosy	@ncoghlan, @pitrou, @vstinner, @rbtcollins, @serhiy-storchaka
Files	12958_aix.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-08-03.21:16:56.938>
created_at = <Date 2014-09-12.18:44:13.762>
labels = ['type-bug', 'library']
title = 'test_socket failure on AIX'
updated_at = <Date 2015-08-03.21:16:56.937>
user = 'https://bugs.python.org/DavidEdelsohn'

bugs.python.org fields:

activity = <Date 2015-08-03.21:16:56.937>
actor = 'r.david.murray'
assignee = 'none'
closed = True
closed_date = <Date 2015-08-03.21:16:56.938>
closer = 'r.david.murray'
components = ['Library (Lib)']
creation = <Date 2014-09-12.18:44:13.762>
creator = 'David.Edelsohn'
dependencies = []
files = ['36612']
hgrepos = []
issue_num = 22397
keywords = ['patch']
message_count = 9.0
messages = ['226837', '227944', '227948', '228000', '228005', '228011', '247908', '247909', '247953']
nosy_count = 8.0
nosy_names = ['ncoghlan', 'pitrou', 'vstinner', 'baikie', 'rbcollins', 'python-dev', 'serhiy.storchaka', 'David.Edelsohn']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue22397'
versions = ['Python 3.3', 'Python 3.4', 'Python 3.5']

[DavidEdelsohn]

AIX has the same test_socket problem with FDPassSeparate as Darwin in bpo-12958 so skip some tests.

[serhiy-storchaka]

LGTM.

[serhiy-storchaka]

See also bpo-11192.

[vstinner]

Current test_socket failures on PPC64 AIX 3.x buildbot:

http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/2672/steps/test/logs/stdio

======================================================================
ERROR: testFDPassSeparateMinSpace (test.test_socket.RecvmsgSCMRightsStreamTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 266, in _tearDown
    raise exc
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 278, in clientRun
    test_func()
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2778, in _testFDPassSeparateMinSpace
    array.array("i", [fd1]))]),
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 1896, in sendmsgToServer
    *(args + self.sendmsg_to_server_defaults[len(args):]))
OSError: [Errno 22] Invalid argument

======================================================================
ERROR: testFDPassSeparate (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 266, in _tearDown
    raise exc
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 278, in clientRun
    test_func()
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2754, in _testFDPassSeparate
    array.array("i", [fd1]))]),
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 1896, in sendmsgToServer
    *(args + self.sendmsg_to_server_defaults[len(args):]))
OSError: [Errno 22] Invalid argument

======================================================================
ERROR: testFDPassSeparateMinSpace (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 266, in _tearDown
    raise exc
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 278, in clientRun
    test_func()
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2778, in _testFDPassSeparateMinSpace
    array.array("i", [fd1]))]),
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 1896, in sendmsgToServer
    *(args + self.sendmsg_to_server_defaults[len(args):]))
OSError: [Errno 22] Invalid argument

======================================================================
FAIL: testIPv4toString (test.test_socket.GeneralModuleTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 976, in testIPv4toString
    assertInvalid(f, '0.0.0.')
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 968, in <lambda>
    (OSError, ValueError), func, a
AssertionError: (<class 'OSError'>, <class 'ValueError'>) not raised by inet_aton

======================================================================
FAIL: testIPv6toString (test.test_socket.GeneralModuleTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 1033, in testIPv6toString
    assertInvalid('1:2:3:4:5:6:7:8:')
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 1011, in <lambda>
    (OSError, ValueError), f, a
AssertionError: (<class 'OSError'>, <class 'ValueError'>) not raised by <lambda>

======================================================================
FAIL: testFDPassSeparate (test.test_socket.RecvmsgSCMRightsStreamTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2742, in testFDPassSeparate
    maxcmsgs=2)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2669, in checkRecvmsgFDs
    self.assertEqual(msg, MSG)
AssertionError: b'' != b'Michael Gilfix was here\xe1\x88\xb4\r\n'

======================================================================
FAIL: testFDPassSeparateMinSpace (test.test_socket.RecvmsgSCMRightsStreamTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2766, in testFDPassSeparateMinSpace
    maxcmsgs=2, ignoreflags=socket.MSG_CTRUNC)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2669, in checkRecvmsgFDs
    self.assertEqual(msg, MSG)
AssertionError: b'' != b'Michael Gilfix was here\xe1\x88\xb4\r\n'

======================================================================
FAIL: testFDPassSeparate (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2742, in testFDPassSeparate
    maxcmsgs=2)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2669, in checkRecvmsgFDs
    self.assertEqual(msg, MSG)
AssertionError: b'' != b'Michael Gilfix was here\xe1\x88\xb4\r\n'

======================================================================
FAIL: testFDPassSeparateMinSpace (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2766, in testFDPassSeparateMinSpace
    maxcmsgs=2, ignoreflags=socket.MSG_CTRUNC)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_socket.py", line 2669, in checkRecvmsgFDs
    self.assertEqual(msg, MSG)
AssertionError: b'' != b'Michael Gilfix was here\xe1\x88\xb4\r\n'

Ran 528 tests in 35.259s

FAILED (failures=6, errors=4, skipped=94)

[vstinner]

Failing tests:

• testFDPassSeparateMinSpace (test.test_socket.RecvmsgSCMRightsStreamTest)
• testFDPassSeparate (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
• testFDPassSeparateMinSpace (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
• testFDPassSeparate (test.test_socket.RecvmsgSCMRightsStreamTest)
• testFDPassSeparateMinSpace (test.test_socket.RecvmsgSCMRightsStreamTest)
• testFDPassSeparate (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
• testFDPassSeparateMinSpace (test.test_socket.RecvmsgIntoSCMRightsStreamTest)

It looks like the failure come from code like:

self.sendmsgToServer([MSG], [(socket.SOL_SOCKET,
                              socket.SCM_RIGHTS,
                              array.array("i", [fd0])),

and

self.doRecvmsg(self.serv_sock, len(MSG),
               socket.CMSG_SPACE(SIZEOF_INT) + socket.CMSG_LEN(SIZEOF_INT)),

In clear, there are errors on tests sending file descriptors through UNIX sockets. So this issue is a duplicate of bpo-20718, but we may keep this one for AIX, and bpo-20718 for OpenBSD.

It looks possible to send a file descriptor between processes but using a pipe with I_SENDFD and I_RECVFD ioctl. See:
http://www-01.ibm.com/support/knowledgecenter/ssw_aix_71/com.ibm.aix.commtrf2/I_SENDFD.htm
http://www-01.ibm.com/support/knowledgecenter/ssw_aix_71/com.ibm.aix.commtrf2/I_RECVFD.htm
http://search.cpan.org/~fractal/AnyEvent-FDpasser-0.3.0/lib/AnyEvent/FDpasser.pm

[vstinner]

The testFDPassSimple pass: it's possible to pass a file descriptor between two processes. Hum. The testMultipleFDPass test also pass: it's possible to pass multiple file descriptors. Both tests use SCM_RIGHTS. The testCMSG_SPACE test also pass.

It looks like the difference between testMultipleFDPass and testFDPassSeparate is the number of "cmsgs" ("access ancillary data"): 1 cmsg for testMultipleFDPass, two for testFDPassSeparate.

I don't understand if it is a bug in the Python socket.sendmsg() function serializing ancillary data, or "just" if the AIX kernel does not support multiple ancillary data for sendmsg().

@david: any idea? Did you ask IBM?

[python-dev]

New changeset e48826904e63 by Robert Collins in branch '3.4':
Issue bpo-22397: Skip failing tests on AIX
https://hg.python.org/cpython/rev/e48826904e63

New changeset a12d29f87e4c by Robert Collins in branch '3.5':
Issue bpo-22397: Skip failing tests on AIX
https://hg.python.org/cpython/rev/a12d29f87e4c

New changeset 62235755609f by Robert Collins in branch 'default':
Issue bpo-22397: Skip failing tests on AIX
https://hg.python.org/cpython/rev/62235755609f

[rbtcollins]

I've updated the ref in the patch to this bug per Victor's comments, and applied.

I'm not sure of protocol here. Do we close this ticket, or keep it open to analyze the actual cause?

[vstinner]

I'm not interested to investigate the issue on AIX. IMHO the feature is
just not supported by the kernel so there is nothing to investigate except
if you want to hack AIX kernel.