test_getgrouplist may fail on macOS if user in too many groups #79251
Comments
|
It looks like macOS 10.14 Mojave has changed the return value for getgroups(). On 10.13 it returns the set of GIDs give by $ python3 -c "import os; print(os.getgroups())"
[101]
$ id -G
101 503 701 501 12 62 80 502 33 98 100 204 250 395 398This breaks test_posix.py. |
|
Hmm, I'm not seeing that behavior with either a freshly-built top of master 3.8 or with the python.org 3.7.1. $ ./python -c "import os;print(os.getgroups())"
[20, 12, 61, 79, 80, 81, 98, 33, 100, 204, 250, 395, 398, 399]
$ id -G
20 12 61 79 80 81 98 33 100 204 250 395 398 399
nad@harj:~/Projects/PyDev/active/dev/3x/source$ ./python -m test -w test_posix
Run tests sequentially
0:00:00 load avg: 1.64 [1/1] test_posix== Tests result: SUCCESS == 1 test OK. Total duration: 13 sec 938 ms
Tests result: SUCCESS
$ ./python
Python 3.8.0a0 (heads/master:9e95eb0d60, Oct 25 2018, 15:55:56)
[Clang 10.0.0 (clang-1000.11.45.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> ^D |
$ sw_vers
ProductName: Mac OS X
ProductVersion: 10.14
BuildVersion: 18A391 |
|
Seems to fail for me with every version of Python 3 on Mojave. In master, it’s test_getgroups().
Mine is exactly the same. |
|
OK. When you asy "every version of Python 3", are those all versions you've built yourself? If so, what ./configure arguments do you use? |
|
On Oct 25, 2018, at 13:17, Ned Deily <report@bugs.python.org> wrote:
Yes, built myself from source (both .tar.gz and git tag). I don’t use any configure arguments, just ./configure && make && make test |
|
I've also tried building on a vanilla-ish 10.8 VM, both without and with installing the /usr/include headers, and I still don't see the failure you're seeing. If you look at Modules/posixmodule.c, you'll see there's a fair amount of Apple-specifc or -caused hackery to deal with past and current getgroups() anomalies. Perhaps you could step through it on your system and see if it's obvious what's happening. Also, looking back through previous issues with getgroups, in bpo-29562 Ronald noted: "Note that the result of getgroups(2) is fixed on login, while "id -G" reflects the current state of the user database on macOS. Could this explain this failure? That is, have you tried logging out and in again before running the test suite?" |
|
"on a vanilla-ish 10.8 VM" ?? I meant 10.14, of course. |
|
FWIW I don't see the problem either, VM running 10.14.1 (beta) with Python 3.7.0. |
|
We're wondering if it could be a weird interaction with Active Directory. This is my work laptop, so it's all integrated with LDAP and whatnot. I don't have Mojave on my personal laptop yet (maybe this weekend). I'm guessing that whatever corporate integration is going on is messing with getgroups(2). Brett said he vaguely remembers something similar, but I don't remember seeing this problem on High Sierra on the same laptop. I'm beta testing Mojave internally, so maybe this is just a weirdism I should report to our IT. |
|
It very well could have something to do with Active Directory support. Here's a little getgroups test I used a while back for a previous getgroups issue. You should see something similar in the output depending on the number of groups in your id. If it fails, we can eliminate Python from the equation. #include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
int main(){
gid_t grouplist[32];
int n;
int gidsetsize;
for(gidsetsize = 0; gidsetsize < 22; ++gidsetsize)
{
n = getgroups(gidsetsize, grouplist);
printf("calling grouplist with gidsetsize = %i, returns %i\n", gidsetsize, n);
}
exit(0);
}calling grouplist with gidsetsize = 0, returns 14 |
|
Also, I'm assuming you've tried rebooting your system? :) |
|
Yes, I've rebooted :) I've modified your C program a little and here's the code and output. #include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
int main()
{
gid_t grouplist[32];
int n;
int gidsetsize;
for (gidsetsize = 0; gidsetsize < 22; ++gidsetsize) {
n = getgroups(gidsetsize, grouplist);
printf("calling grouplist with gidsetsize = %i, returns %i: ", gidsetsize, n);
for (int j = 0; j < n; j++) {
printf("%i ", grouplist[j]);
}
printf("\n");
}
exit(0);
}calling grouplist with gidsetsize = 0, returns 15: -483891656 32766 -483891672 32766 -483891728 32766 427353334 1 -483891696 32766 0 0 0 0 -483891672 % id -G I've also made a small change to test_posix.py: diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py
index 86c04b9f32..5074b45fc0 100644
--- a/Lib/test/test_posix.py
+++ b/Lib/test/test_posix.py
@@ -1047,8 +1047,9 @@ class PosixTester(unittest.TestCase):
# groups, ignoring order, duplicates, and the effective gid.
# python/cpython#55031/#26944 - It is implementation defined whether
# posix.getgroups() includes the effective gid.
- symdiff = idg_groups.symmetric_difference(posix.getgroups())
- self.assertTrue(not symdiff or symdiff == {posix.getegid()})
+ groups = posix.getgroups()
+ symdiff = idg_groups.symmetric_difference(groups)
+ self.assertTrue(not symdiff or symdiff == {posix.getegid()}, (idg_groups, groups, symdiff))
# tests for the posix *at functions follow
But when I run ./python.exe -m test test.test_posix here's what I get:Run tests sequentially
0:00:00 load avg: 1.62 [1/1] test.test_posix
test test.test_posix failed -- Traceback (most recent call last):
File "/Users/bwarsaw/projects/python/cpython/Lib/test/test_posix.py", line 1052, in test_getgroups
self.assertTrue(not symdiff or symdiff == {posix.getegid()}, (idg_groups, groups, symdiff))
AssertionError: False is not true : ({33, 98, 100, 101, 395, 12, 204, 398, 80, 501, 502, 503, 250, 701, 62}, [101], {33, 98, 100, 395, 12, 204, 398, 80, 501, 502, 503, 250, 701, 62})So it seems like getgroups(2) is doing the right thing, but not Python. Weird. |
|
Yeah. Can you check the header file paths that are being used in your compiler invocation, e.g something like: cc -v -c ... |
|
Er, and perhaps the compile and link calls from when Python builds posixmodule.c ? |
% gcc -v gg.c
Apple LLVM version 10.0.0 (clang-1000.11.45.2)
Target: x86_64-apple-darwin18.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.14.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name gg.c -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -fno-strict-return -masm-verbose -munwind-tables -target-cpu penryn -dwarf-column-info -debugger-tuning=lldb -target-linker-version 409.12 -v -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.0 -fdebug-compilation-dir /Users/bwarsaw/projects/python -ferror-limit 19 -fmessage-length 131 -stack-protector 1 -fblocks -fencode-extended-block-signature -fobjc-runtime=macosx-10.14.0 -fmax-type-align=16 -fdiagnostics-show-option -o /var/folders/06/yd7czfp11bx1vx4p5dl10v4w000slb/T/gg-a38469.o -x c gg.c
clang -cc1 version 10.0.0 (clang-1000.11.45.2) default target x86_64-apple-darwin18.0.0
#include "..." search starts here:
#include <...> search starts here:
/usr/local/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.0/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
/usr/include
/System/Library/Frameworks (framework directory)
/Library/Frameworks (framework directory)
End of search list.
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -lto_library /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 10.14.0 -o a.out /var/folders/06/yd7czfp11bx1vx4p5dl10v4w000slb/T/gg-a38469.o -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.0/lib/darwin/libclang_rt.osx.a% otool -L a.out So I guess it's interesting that it's searching /usr/local/include, which has some Homebrew headers in it: % ls /usr/local/include All those symlinks point into ../Cellar/ It doesn't look much different than what happens on 10.13. I'll try to see what the paths are when I build Python. |
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -v -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -I. -I./Include -DPy_BUILD_CORE -c ./Modules/posixmodule.c -o Modules/posixmodule.o
Apple LLVM version 10.0.0 (clang-1000.11.45.2)
Target: x86_64-apple-darwin18.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.14.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -disable-free -disable-llvm-verifier -discard-value-names -main-file-name posixmodule.c -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -fno-strict-return -masm-verbose -munwind-tables -target-cpu penryn -dwarf-column-info -debug-info-kind=standalone -dwarf-version=4 -debugger-tuning=lldb -target-linker-version 409.12 -v -coverage-notes-file /Users/bwarsaw/projects/python/cpython/Modules/posixmodule.gcno -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.0 -D NDEBUG -I . -I ./Include -D Py_BUILD_CORE -O3 -Wno-unused-result -Wsign-compare -Wunreachable-code -Wall -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -std=c99 -fdebug-compilation-dir /Users/bwarsaw/projects/python/cpython -ferror-limit 19 -fmessage-length 131 -fwrapv -stack-protector 1 -fblocks -fencode-extended-block-signature -fobjc-runtime=macosx-10.14.0 -fmax-type-align=16 -fdiagnostics-show-option -vectorize-loops -vectorize-slp -o Modules/posixmodule.o -x c ./Modules/posixmodule.c
clang -cc1 version 10.0.0 (clang-1000.11.45.2) default target x86_64-apple-darwin18.0.0
#include "..." search starts here:
#include <...> search starts here:
.
./Include
/usr/local/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.0/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
/usr/include
/System/Library/Frameworks (framework directory)
/Library/Frameworks (framework directory)
End of search list.
./Modules/posixmodule.c:1254:9: warning: code will never be executed [-Wunreachable-code]
PyErr_SetFromErrno(PyExc_OSError);
^~~~~~~~~~~~~~~~~~
1 warning generated. |
|
I really have no idea what's going on. I've looked at posixmodule.c and tried to reproduce as best I can in a standalone C program. In both cases getgroups(0, NULL) returns 15. In the standalone version, when I then call getgroups(15, grouplist), I again get 15 returned and grouplist is properly filled. But in posixmodule.c case, I get 1 back group that second call. I honestly can't see any difference or why this could be happening. |
|
Yep, stepping through Python with lldb, that's what's happening. I know one of my coworkers has been able to reproduce it. I'll chime in next once I upgrade my personal machine and can try it there, since I know it isn't on AD. |
|
Wonderful |
|
bpo-33223 also reports failure of test_posix under 10.13.4 and seems to be related. |
|
I've now updated my personal machine to Mojave and cannot reproduce the failure. I'm going to chalk this one up to some weird corporate active directory or whatnot weirdness. I'd still love to know why the code in Python works differently that the standalone C version, but I don't plan on spending much time on this issue unless new data comes in. |
|
I'm seeing what appears to my uneducated eyes to be the same failure on Mojave, on a brand new machine which is entirely standalone: 12:16:00 0:00:07 load avg: 4.24 [133/416/1] test_posix failed
12:16:00 test test_posix failed -- Traceback (most recent call last):
12:16:00 File "/Users/jenkins/workspace/python-macos-build/Python-3.7.2/Lib/test/test_posix.py", line 1026, in test_getgrouplist
12:16:00 self.assertIn(group, posix.getgrouplist(user, group))
12:16:00 OSError: [Errno 25] Inappropriate ioctl for device
12:16:00 0:00:07 load avg: 4.24 [134/416/1] test_ast passedSystem info: pgabf-macos2:Python-3.7.2 jenkins$ gcc --version pgabf-macos2:Python-3.7.2 jenkins$ system_profiler SPSoftwareDataType Happy to provide any additional info or try out patches if anyone wants. |
|
The test fails for me on Mohave when I build using clang 10.0.0, but passes when I build using gcc 8.3.0.: ProductName: Mac OS X gcc-8 (Homebrew GCC 8.3.0) 8.3.0 Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/c++/4.2.1 |
|
My mistake. It still failed with gcc 8.3.0. |
|
The submitted patch from websurfer5 resolves the issue for me. |
|
Thanks for the analysis and the PR! It turns out the problem is not new to 10.14; as you discovered, it's simply a matter of how may groups a particular user is in and, with all the system-generated groups, that number may vary from release to release and from user to user - which explains why the problem is not always reproducible. I've added a few review comments to the PR. |
ned-deily
changed the title
|
Merged for release in 3.8.0b2 and 3.7.4. |
|
This issue has roamed quite a bit so it is a little bit difficult to tell what problem(s) have been seen here. But, clearly Jeffrey's PR fixes a real and now reproducible problem so I'm declaring victory and have closed this issue now. Thanks everyonw! I am going to leave bpo-33223 open (thanks xtreak!) for further investigation. |
Well done Jeffrey Kintscher! The test was failing randomly for years, I recall many bugs and many people complaining, but nobody succeeded to come up with a fix. Note: this off-by-one looks magic to me, but I'm happy that it works :-) |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: