Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_socket failures on Mac OS X #57167

Closed
ncoghlan opened this issue Sep 11, 2011 · 14 comments
Closed

test_socket failures on Mac OS X #57167

ncoghlan opened this issue Sep 11, 2011 · 14 comments
Assignees
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@ncoghlan
Copy link
Contributor

BPO 12958
Nosy @brettcannon, @ronaldoussoren, @ncoghlan, @ned-deily, @bitdancer, @voidspace

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = 'https://github.com/ncoghlan'
closed_at = <Date 2011-09-19.10:26:46.561>
created_at = <Date 2011-09-11.05:40:43.125>
labels = ['type-bug', 'library']
title = 'test_socket failures on Mac OS X'
updated_at = <Date 2014-02-01.10:21:12.638>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2014-02-01.10:21:12.638>
actor = 'ned.deily'
assignee = 'ncoghlan'
closed = True
closed_date = <Date 2011-09-19.10:26:46.561>
closer = 'python-dev'
components = ['Library (Lib)']
creation = <Date 2011-09-11.05:40:43.125>
creator = 'ncoghlan'
dependencies = []
files = []
hgrepos = ['67']
issue_num = 12958
keywords = []
message_count = 14.0
messages = ['143854', '143860', '143863', '144241', '144248', '144257', '144264', '159694', '187635', '187637', '187642', '187643', '187646', '209884']
nosy_count = 8.0
nosy_names = ['brett.cannon', 'ronaldoussoren', 'ncoghlan', 'baikie', 'ned.deily', 'r.david.murray', 'michael.foord', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue12958'
versions = ['Python 3.3', 'Python 3.4']

@ncoghlan
Copy link
Contributor Author

Several of the new socket tests introduced in bpo-6560 are failing on the Mac OS X buildbots. These appear to be due to platform problems rather than anything in the test suite or the new sendmsg/recvmsg code, but it would be good if a developer on Mac OS X could take a closer look at them.

To avoid polluting the buildbot results, the affected tests will be flagged as expected failures when run on Mac OS X (this isn't in the repo yet, but once it is, the tests will be marked with "anticipate_failure" in the test_socket code).

@ncoghlan ncoghlan added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Sep 11, 2011
@ncoghlan
Copy link
Contributor Author

First attempt didn't quite work - the FD passing tests somehow seem to be reporting both 'ERROR' *and* 'expected failure', which is causing the test overall to remain red.

http://www.python.org/dev/buildbot/all/builders/AMD64%20Snow%20Leopard%202%20custom/builds/41/steps/test/logs/stdio

@ncoghlan
Copy link
Contributor Author

Ah, I believe I see why the expected failure isn't working properly - the failing testFDPass* tests are causing the subsequent tear down code to also fail.

@ncoghlan
Copy link
Contributor Author

OK, I've now looked into *why* the socket tests are throwing errors in tearDown, and it has to do with the way the threaded client/server tests in test_socket are set up.

Specifically, ThreadableTest uses tearDown to reraise any exception raised in the client thread, and these are therefore outside the scope of the "expectedFailure" suppression in unittest.

Now that I've tracked this down, it would be fairly straightforward to fix this specifically within test_socket.ThreadableTest by appropriately adjusting the definition of ThreadableTest.clientRun to discard exceptions encountered in tests flagged as expected failures.

However, I'm wondering if that's the right thing to do. Perhaps it would make more sense to change unittest itself so that "expectedFailure" also suppresses tearDown errors. It doesn't seem all that unusual for a known failing test to also cause problems for the tearDown code.

Added Michael to the nosy list to ask for his advice/opinion. In the meantime, I'll work on a patch that adjusts ThreadableTest directly.

@ncoghlan ncoghlan self-assigned this Sep 18, 2011
@voidspace
Copy link
Contributor

See bpo-10548. There is some resistance to expectedFailure masking errors in setUp/tearDown as these aren't the place where you would normally expect the expected failure...

@ncoghlan
Copy link
Contributor Author

OK, I'll just deal with the problem directly in test_socket then.

It looks like my latest attempt (suppressing unittest._ExpectedFailure in test_socket.ThreadableTest.clientRun) did the trick, so I'll push the updated tests some time this evening:
http://www.python.org/dev/buildbot/all/builders/AMD64%20Snow%20Leopard%202%20custom/builds/44/steps/test/logs/stdio

@python-dev
Copy link
Mannequin

python-dev mannequin commented Sep 19, 2011

New changeset a4e4facad164 by Nick Coghlan in branch 'default':
Close bpo-12958 by flagging expected failures in test_socket on Mac OS X
http://hg.python.org/cpython/rev/a4e4facad164

@python-dev python-dev mannequin closed this as completed Sep 19, 2011
@python-dev
Copy link
Mannequin

python-dev mannequin commented Apr 30, 2012

New changeset e64bec91ac91 by Richard Oudkerk in branch 'default':
Issue bpo-14669: Skip multiprocessing connection pickling test on MacOSX
http://hg.python.org/cpython/rev/e64bec91ac91

@ronaldoussoren
Copy link
Contributor

Not sure what changed, but these tests are reported as failures when I run 'make test' and not as expected failures.

@bitdancer
Copy link
Member

Ronald: in 3.4 only? If so I wonder if the subtests patch had an unintended side effect...I'm pretty sure it touched the expected failure machinery somehow.

@ronaldoussoren
Copy link
Contributor

Yes, only on 3.4:

======================================================================
ERROR: testFDPassSeparate (test.test_socket.RecvmsgSCMRightsStreamTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Users/ronald/Projects/python/rw/default/Lib/test/test_socket.py", line 261, in _tearDown
    raise exc
  File "/Users/ronald/Projects/python/rw/default/Lib/test/test_socket.py", line 273, in clientRun
    test_func()
  File "/Users/ronald/Projects/python/rw/default/Lib/test/test_socket.py", line 2644, in _testFDPassSeparate
    array.array("i", [fd1]))]),
  File "/Users/ronald/Projects/python/rw/default/Lib/test/test_socket.py", line 1790, in sendmsgToServer
    *(args + self.sendmsg_to_server_defaults[len(args):]))
OSError: [Errno 22] Invalid argument

(And simular for the other 3 marked socket tests)

@ncoghlan
Copy link
Contributor Author

It's entirely possible the hack in test_socket (where it catches unittest._ExpectedFailure to suppress the issues in the tearDown code after the test fails) were invalidated by the subtest changes.

@ronaldoussoren
Copy link
Contributor

FWIW I've filed a bug report about the behavior of sendmsg with Apple, the RADAR number is 13716133.

@ned-deily
Copy link
Member

FYI, the OS X platform bugs causing the three "send timeout" test failures were fixed in OS X 10.7 causing those test cases to have previously silently skipped "unexpected successes". See bpo-20474 for more details.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

5 participants