New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test_getgroups of test_posix fails (on OS X 10.10) #73748
Comments
When I run test.test_posix.PosixTester.test_getgroups on my Mac OS X system, it fails: % ./python.exe -m unittest -v test.test_posix.PosixTester.test_getgroups ====================================================================== Traceback (most recent call last):
File "/Users/jdlh/workspace/cpython/Lib/test/test_posix.py", line 824, in test_getgroups
self.assertTrue(not symdiff or symdiff == {posix.getegid()})
AssertionError: False is not true Ran 1 test in 0.013s FAILED (failures=1) Details of my system: % id -G % ./python.exe -c 'import grp,os; g={i: (n, p, i, mem) for (n, p, i, mem) in grp.getgrall()}; print(sorted([(i, g[i][0]) for i in os.getgroups()]) )' So the difference, which triggers the test failure, is that id -G is returning groups (701, 'com.apple.sharepoint.group.1'), and (398, 'com.apple.access_screensharing'), while posix.getgroups() is not. I do not yet understand why. Others say this test works on their OS X 10.10 system, so maybe it's triggered by something in my environment. Also: python3.6 from MacPorts, and python2.7 from MacPorts, return the same set of groupids as does the dev build of python3.7. This bug affects the same test, and the same posix.getgroups() call, as http://bugs.python.org/issue17557 "test_getgroups of test_posix can fail on OS X 10.8 if more than 16 groups" (2013-2014, closed). But I think it is a different problem: bpo-17557 is related to how posix.getgroups() deals with large numbers of groups, and it is fixed. I would appreciate help in getting this test to pass. Maybe my environment is wrong, in which case I should fix my environment. But maybe the cpython code is sensitive to some detail of my environment, in which case perhaps I should fix the cpython code. |
I have pushed a branch for this issue to my cpython fork: https://github.com/JDLH/cpython/tree/bpo-29562_failing_test_getgroups_on_os_x It modifies test_getgroups in test_posix.py to give better diagnostics in the event of a test failure. It says specifically which groups were in id -G, and posix.getgroups(), but not in the other. % ./python.exe -m unittest -v test.test_posix.PosixTester.test_getgroups ====================================================================== Traceback (most recent call last):
File "/Users/jdlh/workspace/cpython/Lib/test/test_posix.py", line 841, in test_getgroups
self.assertEqual(len(symdiff), 0, msg)
AssertionError: 2 != 0 : id -G and posix.groups() should have zero difference.
Groups in id -G but not posix.groups(): [(701, 'com.apple.sharepoint.group.1'), (398, 'com.apple.access_screensharing')]
Groups in posix.groups() but not id -G: []
(Effective GID (20) was disregarded.) Ran 1 test in 0.020s I don't think this branch is ready yet to submit to the main codebase, but it may help people diagnose the issue. |
Some diagnosis. Group Its only member was a test user I created as part of screen sharing with Apple Support.
I removed File Sharing for this user's home directory.
Having done that, the group
Interestingly,
Earlier, group That makes me think that the behaviour of getgroups (2) in Mac OS is behaving differently than we expect.
I don't know how to determine if my copy of Mac OS X 10.10 was complied with either of these two macros. On my system, I chased NGROUPS_MAX down to /usr/include/sys/syslimits.h:84, where it is set to 16. That is more than the number of groups
This is 12 groups, whereas before it was 13 groups (see my message from 2017-02-15 02:03). This is unsurprising. However, the number of groups returned by posix.getgroups() has also shrunk by 1:
Notice that group (395, 'com.apple.access_ftp') is no longer being returned by os.getgroups(). This is as a consequence of a different group being deleted. The test_getgroups comment asserts: "# 'id -G' and 'os.getgroups()' should return the same groups, ignoring order, duplicates, and the effective gid." https://github.com/python/cpython/blob/master/Lib/test/test_posix.py#L819-L820 I'm getting skeptical about that claim. Does Mac OS X actually guarantee that 'id -G' and 'getgroups(2)' return the same groups? |
I guess I didn't state the things I find odd about what the new test_getgroups results.
|
The Mac OS 10.10 man page for initgroups(3) says: "Processes should not use the group ID numbers from getgroups(2) to determine a user's group membership. The list obtained from getgroups() may only be a partial list of a user's group membership. Membership checks should use the mbr_gid_to_uuid(3), mbr_uid_to_uuid(3), and mbr_check_membership(3) functions." When the man page says, "The list obtained from getgroups() may only be a partial list of a user's group membership.", and the list from Or, should we consider rewriting os_getgroups_impl() to use a Mac-specific implementation on Mac OS X? |
Note that the result of getgroups(2) is fixed on login, while "id -G" reflects the current state of the user database on macOS. Could this explain this failure? That is, have you tried logging out and in again before running the test suite? |
Wow, that's interesting! Thank you for this information. The test code for test_getgroups does not mention this interaction. I can certainly see how it could affect the test. Maybe it should be added? Since I last tried that test, I've logged out and restarted several times, and changed OS to Mac OS X 10.11 El Capitan. Nothing like changing several independent variables at once while diagnosing! I will try the test again and report back. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: