test_getgroups of test_posix can fail on OS X 10.8 if more than 16 groups

[ned-deily]

BPO	17557
Nosy	@jcea, @ronaldoussoren, @tebeka, @ned-deily
Files	getgroups_osx.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ned-deily'
closed_at = <Date 2013-08-02.04:43:31.814>
created_at = <Date 2013-03-27.09:55:41.427>
labels = ['OS-mac']
title = 'test_getgroups of test_posix can fail on OS X 10.8 if more than 16 groups'
updated_at = <Date 2014-02-11.22:04:35.550>
user = 'https://github.com/ned-deily'

bugs.python.org fields:

activity = <Date 2014-02-11.22:04:35.550>
actor = 'ned.deily'
assignee = 'ned.deily'
closed = True
closed_date = <Date 2013-08-02.04:43:31.814>
closer = 'ned.deily'
components = ['macOS']
creation = <Date 2013-03-27.09:55:41.427>
creator = 'ned.deily'
dependencies = []
files = ['30816']
hgrepos = []
issue_num = 17557
keywords = ['patch']
message_count = 9.0
messages = ['185319', '192399', '192436', '192513', '192525', '194149', '194150', '210959', '211028']
nosy_count = 6.0
nosy_names = ['jcea', 'ronaldoussoren', 'tebeka', 'ned.deily', 'python-dev', 'mlen']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue17557'
versions = ['Python 2.7', 'Python 3.3', 'Python 3.4']

[ned-deily]

Due to a change in behavior for OS X 10.8 (seen with 10.8.3), the code added to posixmodule.c in bpo-7900 to handle an unlimited number of groups no longer works. The code depends on the documented behavior of getgroups(2) failing with EINVAL when the grouplist array in the call is too small to hold all groups. This works correctly for 10.6 and 10.7. Currently in 10.8, such a call succeeds and truncates to the first grouplist-size groups. The getgroups function could probably be modified to always call getgroups(0) first to get the real length. But it seems to be a clear regression in 10.8 and breaks existing code. I've opened a bug report with Apple about it. I'll plan to keep this incident open until I hear something back from them.

[ronaldoussoren]

Have you heard anything from Apple? If you have a radar number I could try to ping someone at Apple.

I'd be in favor of working around this issue in posixmodule.c if they don't respond or if they won't fix the issue in 10.8 (and I wouldn't be surprised if they won't).

[mlen]

I attached a patch implementing a workaround for getgroups(2) call. When compiled for OS X it first calls getgroups with 0 as the first argument to get number of groups and then in subsequent call the code fetches the gids.

This patch applies cleanly on 2.7 and 3.x branches.

[ned-deily]

Thanks for the patch, Mateusz. That basic approach should work on all current releases (although it would be nice if Apple also fixed the problem). I'll test it on the various releases. If you haven't already, please submit a contributor form as described in the developer's guide (http://docs.python.org/devguide/patch.html#licensing).

[mlen]

I signed it today.

[python-dev]

New changeset 6d3b7e0559a0 by Ned Deily in branch '2.7':
Issue bpo-17557: Fix os.getgroups() to work with the modified behavior of
http://hg.python.org/cpython/rev/6d3b7e0559a0

New changeset 0a4afa8833b5 by Ned Deily in branch '3.3':
Issue bpo-17557: Fix os.getgroups() to work with the modified behavior of
http://hg.python.org/cpython/rev/0a4afa8833b5

New changeset 634a8e8816d4 by Ned Deily in branch 'default':
Issue bpo-17557: merge from 3.3
http://hg.python.org/cpython/rev/634a8e8816d4

[ned-deily]

Committed patch (with slight modifications) for release in 2.7.6, 3.3.3, and 3.4.0. Thanks again, Mateusz.

[tebeka]

I still see this in 3.4rc1

======================================================================
FAIL: test_getgroups (test.test_posix.PosixTester)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/private/tmp/Python-3.4.0rc1/Lib/test/test_posix.py", line 780, in test_getgroups
    set(posix.getgroups() + [posix.getegid()]))
AssertionError: Items in the second set but not the first:
33
100
204
398
399

[ned-deily]

Miki, what is the output of the following on the failing system?

id -G && sw_vers