test_socket error on AIX

[sable]

BPO	11192
Nosy	@ncoghlan, @giampaolo, @aixtools
PRs	bpo-11192: Fix test_socket.py for AIX #8954 bpo-34579: Modify test_embed DEFAULT_CONFIG for AIX #9063 [3.7] bpo-11192: Fix test_socket.py for AIX (GH-8954) #14237

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-12-28.14:08:27.417>
created_at = <Date 2011-02-11.15:05:50.784>
labels = ['3.7', '3.8', 'type-bug', 'tests']
title = 'test_socket error on AIX'
updated_at = <Date 2019-06-19.14:46:58.677>
user = 'https://bugs.python.org/sable'

bugs.python.org fields:

activity = <Date 2019-06-19.14:46:58.677>
actor = 'Michael.Felt'
assignee = 'none'
closed = True
closed_date = <Date 2018-12-28.14:08:27.417>
closer = 'ncoghlan'
components = ['Tests']
creation = <Date 2011-02-11.15:05:50.784>
creator = 'sable'
dependencies = []
files = []
hgrepos = []
issue_num = 11192
keywords = ['patch']
message_count = 12.0
messages = ['128400', '128671', '191132', '191418', '271277', '310145', '310165', '323533', '323953', '332527', '337634', '337699']
nosy_count = 7.0
nosy_names = ['ncoghlan', 'sable', 'giampaolo.rodola', 'alef', 'David.Edelsohn', 'delhallt', 'Michael.Felt']
pr_nums = ['8954', '9063', '14237']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue11192'
versions = ['Python 2.7', 'Python 3.7', 'Python 3.8']

[sable]

I have the following error in test_socket on AIX:

======================================================================
FAIL: testGetaddrinfo (main.GeneralModuleTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "./Lib/test/test_socket.py", line 640, in testGetaddrinfo
    self.assertEqual(socktype, socket.SOCK_STREAM)
AssertionError: 0 != 1

---

Ran 158 tests in 16.904s

FAILED (failures=1, skipped=4)
Traceback (most recent call last):
  File "./Lib/test/test_socket.py", line 2012, in <module>
    test_main()
  File "./Lib/test/test_socket.py", line 2008, in test_main
    support.run_unittest(*tests)
  File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/support.py", line 1145, in run_unittest
    _run_suite(suite)
  File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/support.py", line 1128, in _run_suite
    raise TestFailed(err)
test.support.TestFailed: Traceback (most recent call last):
  File "./Lib/test/test_socket.py", line 640, in testGetaddrinfo
    self.assertEqual(socktype, socket.SOCK_STREAM)
AssertionError: 0 != 1

[119732 refs]

I haven't investigated yet.

[sable]

I am not sure this is directly related, but test_socket.py will hang forever on AIX 6.1 when this script is run as part of "make buildbottest".

It won't hang when running the script alone.

[delhallt]

not a python problem, see closed aix issue http://www-01.ibm.com/support/docview.wss?uid=isg1IZ57712

[alef]

Same error with Python 2.7.3 on AIX 6.1

[aixtools]

FYI: test fails on AIX 5.3 TL7 SP0 (as expected. The IBM document refers to 5.3 TL8)

On AIX 6.1 TL9 SP5 (and probably earlier) the test passes. IMHO - can be closed (and README.AIX can be updated).

michael@x071:[/data/prj/aixtools/python/python-2.7.11.5/Lib/test]oslevel -q -s | head -5
Known Service Packs
-------------------
6100-09-05-1524
6100-09-04-1441

michael@x071:[/data/prj/aixtools/python/python-2.7.11.5/Lib/test]../../python test_socket.py
...
----------------------------------------------------------------------
Ran 124 tests in 15.860s

OK (skipped=7)

On AIX 5.3 (the expected fail...)
root@x064:[/data/prj/aixtools/python/python-2.7.11.5/Lib/test]oslevel -q -s | head -5
Known Service Packs
-------------------
5300-07-00-0000
5300-06-03-0732

root@x064:[/data/prj/aixtools/python/python-2.7.11.5/Lib/test]../../python test_socket.py

...

======================================================================
FAIL: testGetaddrinfo (main.GeneralModuleTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "test_socket.py", line 691, in testGetaddrinfo
    self.assertEqual(socktype, socket.SOCK_STREAM)
AssertionError: 0 != 1

---

Ran 124 tests in 16.765s

FAILED (failures=1, skipped=7)
Traceback (most recent call last):
  File "test_socket.py", line 1804, in <module>
    test_main()
  File "test_socket.py", line 1800, in test_main
    test_support.run_unittest(*tests)
  File "/data/prj/aixtools/python/python-2.7.11.5/Lib/test/test_support.py", line 1428, in run_unittest
    _run_suite(suite)
  File "/data/prj/aixtools/python/python-2.7.11.5/Lib/test/test_support.py", line 1411, in _run_suite
    raise TestFailed(err)
test.test_support.TestFailed: Traceback (most recent call last):
  File "test_socket.py", line 691, in testGetaddrinfo
    self.assertEqual(socktype, socket.SOCK_STREAM)
AssertionError: 0 != 1

[aixtools]

Back to master (python3-3.7)

(note: not your problem - but this link is gone: not a python problem, see closed aix issue http://www-01.ibm.com/support/docview.wss?uid=isg1IZ57712)

Getting this (obviously) on AIX 5.3, but also on AIX 6.1 (TL6), but not on AIX 7.1 TL2: (both gcc and xlc)

testGetSockOpt (test.test_socket.GeneralModuleTests) ... ok
testGetaddrinfo (test.test_socket.GeneralModuleTests) ... FAIL
testHostnameRes (test.test_socket.GeneralModuleTests) ... ok

However, what I get on all three systems (but do not see on the AIX buildbot running on AIX 7.2, and I have no access to AIX 7.2) is:

AIX 6.1:
Ran 669 tests in 25.899s

FAILED (failures=1, errors=114, skipped=119)

AIX 7.1:

Ran 669 tests in 25.917s

FAILED (errors=114, skipped=119)

What all the ERRORS have in common is:

File "/data/prj/python/git/x067-python3-3.7/Lib/test/test_socket.py", line 2373, in setUp
self.serv_sock.settimeout(self.fail_timeout)
AttributeError: 'SendmsgTests' object has no attribute 'serv_sock'

HTH - (and still digging)

[aixtools]

I have tried jumping into the pdp using

but do not come much further in understanding how the expectations expressed in these comments are satisfied:

+2153 # Tests for the sendmsg()/recvmsg() interface. Where possible, the
+2154 # same test code is used with different families and types of socket
+2155 # (e.g. stream, datagram), and tests using recvmsg() are repeated
+2156 # using recvmsg_into().
+2157 #
+2158 # The generic test classes such as SendmsgTests and
+2159 # RecvmsgGenericTests inherit from SendrecvmsgBase and expect to be
+2160 # supplied with sockets cli_sock and serv_sock representing the
+2161 # client's and the server's end of the connection respectively, and
+2162 # attributes cli_addr and serv_addr holding their (numeric where
+2163 # appropriate) addresses.
+2164 #

++ Details ++
14 tests failed again:
test_asyncio test_cmath test_ctypes test_datetime test_distutils
test_httplib test_httpservers test_os test_posix test_socket
test_ssl test_subprocess test_time test_utf8_mode

()
-> super().setUp()
(Pdb) s
--Call--

/data/prj/python/git/x067-python3-3.7/Lib/test/test_socket.py(2187)setUp()
-> def setUp(self):
(Pdb) l
2182
2183 # Time in seconds to wait before considering a test failed, or
2184 # None for no timeout. Not all tests actually set a timeout.
2185 fail_timeout = 3.0
2186
2187 -> def setUp(self):
2188 self.misc_event = threading.Event()
2189 super().setUp()
2190
2191 def sendToServer(self, msg):
2192 # Send msg to the server.
(Pdb) s
/data/prj/python/git/x067-python3-3.7/Lib/test/test_socket.py(2188)setUp()
-> self.misc_event = threading.Event()
(Pdb) self
<test.test_socket.RFC3542AncillaryTest testMethod=testCmsgTruncNoBufSize>
(Pdb) s
--Call--
/data/prj/python/git/x067-python3-3.7/Lib/threading.py(499)init()
-> def __init__(self):
(Pdb) self
<threading.Event object at 0x3035f3d0>
(Pdb) dir(self)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_reset_internal_locks', 'clear', 'isSet', 'is_set', 'set', 'wait']
(Pdb) l
494
495 """
496
497 # After Tim Peters' event class (without is_posted())
498
499 -> def __init__(self):
500 self._cond = Condition(Lock())
501 self._flag = False
502
503 def _reset_internal_locks(self):
504 # private! called by Thread._reset_internal_locks by _after_fork()
(Pdb) n
/data/prj/python/git/x067-python3-3.7/Lib/threading.py(500)init()
-> self._cond = Condition(Lock())
(Pdb) n
/data/prj/python/git/x067-python3-3.7/Lib/threading.py(501)init()
-> self._flag = False
(Pdb) n
--Return--
/data/prj/python/git/x067-python3-3.7/Lib/threading.py(501)init()->None
-> self._flag = False
(Pdb) n
/data/prj/python/git/x067-python3-3.7/Lib/test/test_socket.py(2189)setUp()
-> super().setUp()
(Pdb) l
2184 # None for no timeout. Not all tests actually set a timeout.
2185 fail_timeout = 3.0
2186
2187 def setUp(self):
2188 self.misc_event = threading.Event()
2189 -> super().setUp()
2190
2191 def sendToServer(self, msg):
2192 # Send msg to the server.
2193 return self.cli_sock.send(msg)
2194
(Pdb) s
--Call--
/data/prj/python/git/x067-python3-3.7/Lib/unittest/case.py(453)setUp()
-> def setUp(self):
(Pdb) l
448 called after tearDown on test failure or success.
449
450 Cleanup items are called even if setUp fails (unlike tearDown)."""
451 self._cleanups.append((function, args, kwargs))
452
453 -> def setUp(self):
454 "Hook method for setting up the test fixture before exercising it."
455 pass
456
457 def tearDown(self):
458 "Hook method for deconstructing the test fixture after testing it."
(Pdb) n
/data/prj/python/git/x067-python3-3.7/Lib/unittest/case.py(455)setUp()
-> pass
(Pdb) n
--Return--
/data/prj/python/git/x067-python3-3.7/Lib/unittest/case.py(455)setUp()->None
-> pass
(Pdb) n
--Return--
/data/prj/python/git/x067-python3-3.7/Lib/test/test_socket.py(2189)setUp()->None
-> super().setUp()
(Pdb) l
2184 # None for no timeout. Not all tests actually set a timeout.
2185 fail_timeout = 3.0
2186
2187 def setUp(self):
2188 self.misc_event = threading.Event()
2189 -> super().setUp()
2190
2191 def sendToServer(self, msg):
2192 # Send msg to the server.
2193 return self.cli_sock.send(msg)
2194
(Pdb) s
/data/prj/python/git/x067-python3-3.7/Lib/test/test_socket.py(2375)setUp()
-> print('stop here') #@@@
(Pdb) l
2370
2371 def setUp(self):
2372 import pdb #@@@
2373 pdb.set_trace() #@@@
2374 super().setUp()
2375 -> print('stop here') #@@@
2376 print(self)
2377 super().setUp()
2378 self.serv_sock.settimeout(self.fail_timeout)
2379
2380
(Pdb) dir(self)
['__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_addExpectedFailure', '_addSkip', '_addUnexpectedSuccess', '_baseAssertEqual', '_classSetupFailed', '_cleanup_lock', '_cleanups', '_deprecate', '_diffThreshold', '_feedErrorsToResult', '_formatMessage', '_getAssertEqualityFunc', '_outcome', '_subtest', '_testCmsgTruncNoBufSize', '_testMethodDoc', '_testMethodName', '_testOddCmsgSize', '_testRecvHopLimit', '_testRecvHopLimitCMSG_SPACE', '_testRecvTrafficClassAndHopLimit', '_testRecvTrafficClassAndHopLimitCMSG_SPACE', '_testSecomdCmsgTruncInData', '_testSecondCmsgTrunc0', '_testSecondCmsgTrunc1', '_testSecondCmsgTrunc2Int', '_testSecondCmsgTruncLen0Minus1', '_testSetHopLimit', '_testSetTrafficClassAndHopLimit', '_testSingleCmsgTrunc0', '_testSingleCmsgTrunc1', '_testSingleCmsgTrunc2Int', '_testSingleCmsgTruncInData', '_testSingleCmsgTruncLen0Minus1', '_truncateMessage', '_type_equality_funcs', 'addCleanup', 'addTypeEqualityFunc', 'ancillaryMapping', 'assertAlmostEqual', 'assertAlmostEquals', 'assertCountEqual', 'assertDictContainsSubset', 'assertDictEqual', 'assertEqual', 'assertEquals', 'assertFalse', 'assertGreater', 'assertGreaterEqual', 'assertIn', 'assertIs', 'assertIsInstance', 'assertIsNone', 'assertIsNot', 'assertIsNotNone', 'assertLess', 'assertLessEqual', 'assertListEqual', 'assertLogs', 'assertMultiLineEqual', 'assertNotAlmostEqual', 'assertNotAlmostEquals', 'assertNotEqual', 'assertNotEquals', 'assertNotIn', 'assertNotIsInstance', 'assertNotRegex', 'assertNotRegexpMatches', 'assertRaises', 'assertRaisesRegex', 'assertRaisesRegexp', 'assertRegex', 'assertRegexpMatches', 'assertSequenceEqual', 'assertSetEqual', 'assertTrue', 'assertTupleEqual', 'assertWarns', 'assertWarnsRegex', 'assert_', 'checkFlags', 'checkHopLimit', 'checkHopLimitTruncatedHeader', 'checkRecvmsgAddress', 'checkTrafficClassAndHopLimit', 'checkTruncatedSecondHeader', 'countTestCases', 'debug', 'defaultTestResult', 'doCleanups', 'doRecvmsg', 'fail', 'failIf', 'failIfAlmostEqual', 'failIfEqual', 'failUnless', 'failUnlessAlmostEqual', 'failUnlessEqual', 'failUnlessRaises', 'fail_timeout', 'failureException', 'hop_limit', 'id', 'longMessage', 'maxDiff', 'misc_event', 'msg_flags_common_set', 'msg_flags_common_unset', 'msg_flags_eor_indicator', 'msg_flags_non_eor_indicator', 'name', 'registerRecvmsgResult', 'run', 'sendToServer', 'sendmsgToServer', 'sendmsg_to_server_defaults', 'setUp', 'setUpClass', 'shortDescription', 'skipTest', 'subTest', 'tearDown', 'tearDownClass', 'testCmsgTruncNoBufSize', 'testOddCmsgSize', 'testRecvHopLimit', 'testRecvHopLimitCMSG_SPACE', 'testRecvTrafficClassAndHopLimit', 'testRecvTrafficClassAndHopLimitCMSG_SPACE', 'testSecomdCmsgTruncInData', 'testSecondCmsgTrunc0', 'testSecondCmsgTrunc1', 'testSecondCmsgTrunc2Int', 'testSecondCmsgTruncLen0Minus1', 'testSetHopLimit', 'testSetTrafficClassAndHopLimit', 'testSingleCmsgTrunc0', 'testSingleCmsgTrunc1', 'testSingleCmsgTrunc2Int', 'testSingleCmsgTruncInData', 'testSingleCmsgTruncLen0Minus1', 'traffic_class']
(Pdb) self
<test.test_socket.RFC3542AncillaryTest testMethod=testCmsgTruncNoBufSize>

[aixtools]

The original error reported was fixed by AIX - in AIX 6.1.

It will remain unresolved in AIX 5.3.

There are currently two other errors in test_socket.

FAIL: test_getnameinfo_ipv6_scopeid_symbolic (test.test_socket.GeneralModuleTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/data/prj/python/git/python3-3.7/Lib/test/test_socket.py", line 1649, in test_getnameinfo_ipv6_scopeid_symbolic
    self.assertEqual(nameinfo, ('ff02::1de:c0:face:8d%' + test_interface, '1234'))
AssertionError: Tuples differ: ('ff02::1de:c0:face:8d', '1234') != ('ff02::1de:c0:face:8d%en0', '1234')

First differing element 0:
'ff02::1de:c0:face:8d'
'ff02::1de:c0:face:8d%en0'

• ('ff02::1de:c0:face:8d', '1234')
  + ('ff02::1de:c0:face:8d%en0', '1234')
  ? ++++

I assume the capital D in the argument in the test:
sockaddr = ('ff02::1de:c0:face:8D', 1234, 0, ifindex) # Note capital letter D.

is meant to trigger getting the IPv6 zoneinfo based on ifindex.

AIX does have a routine to strip zoneinfo off an IPv6 address, but the getnameinfo() and getaddrinfo() do not support this.

I can create a new issue specifically for IPv6 zone id issue, or I can add a PR here.

Please advise (the PR is merely adding "skipUnless" logic to test_socket.py.

ERROR: test_getaddrinfo_ipv6_scopeid_symbolic

[aixtools]

I nearly have a patch - to resolve all the remaining test_socket() issues with AIX. However, this (new?) issue remains:

        f = lambda a: inet_pton(AF_INET6, a)
        assertInvalid = lambda a: self.assertRaises(
            (OSError, ValueError), f, a
        )

>>> f('1:2:3:4:5:6:7:8')
b'\x00\x01\x00\x02\x00\x03\x00\x04\x00\x05\x00\x06\x00\x07\x00\x08'
>>> f('1:2:3:4:5:6:7:8:')
b'\x00\x01\x00\x02\x00\x03\x00\x04\x00\x05\x00\x06\x00\x07\x00\x08'
>>> f('1::')
b'\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
>>> f('1:2:3:4:5:6:7:8::')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <lambda>
OSError: illegal IP address string passed to inet_pton

>>> f('1:2:3:4:5:6:7')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <lambda>
OSError: illegal IP address string passed to inet_pton

Is, maybe, the last test meant to be '::' on the end, rather than ':'?

[ncoghlan]

New changeset 5661459 by Nick Coghlan (Michael Felt) in branch 'master':
bpo-11192: Skip unsupported cases in test_socket on AIX (GH-8954)
5661459

[aixtools]

Could this also be backported to Version 3.7 and 3.6 (I do not expect it to be backported to 2.7, but I had mistakenly removed it 2.7 when I changed it to 3.8 - and should have added 3.6 and 3.7 then).

[aixtools]

On 10/03/2019 19:40, Michael Felt wrote:

Michael Felt <aixtools@felt.demon.nl> added the comment:

Could this also be backported to Version 3.7 and 3.6 (I do not expect it to be backported to 2.7, but I had mistakenly removed it 2.7 when I changed it to 3.8 - and should have added 3.6 and 3.7 then).
Likewise: As 3.6 is in security mode - I guess only 3.7 then.

----------
versions: +Python 2.7, Python 3.6, Python 3.7

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue11192\>

---