os/exec: “operation not permitted” in TestCredentialNoSetGroups

[bcmills]

TestCredentialNoSetGroups fails reliably on my (Debian-based) Linux machine.

~/go/src$ go version
go version devel +62f0127d81 Fri Apr 6 19:42:41 2018 +0000 linux/amd64
~/go/src$ ./all.bash
[…]
--- FAIL: TestCredentialNoSetGroups (0.01s)
        exec_posix_test.go:44: Failed to run command: fork/exec /tmp/go-build301742174/b642/exec.test: operation not permitted
FAIL
FAIL    os/exec 1.028s

(CC: @ianlancetaylor, @walac)

[mvdan]

What is special about this machine? The test works fine on my Arch Linux box. Are you running the test as your regular user? Does running the test alone fail too?

[bcmills]

It's a company-owned virtualized machine running a variant of Debian Testing. I'm sure there are many special things about it, but I'm sure I can't enumerate them. 🙂

I am running the test as my regular user. The test fails in isolation, and the test binary also fails if I run in directly.

--test.v output attached.

Details

~/go/src$ go test -c os/exec
~/go/src$ ./exec.test --test.v
=== RUN   TestDedupEnv
--- PASS: TestDedupEnv (0.00s)
=== RUN   TestPrefixSuffixSaver
--- PASS: TestPrefixSuffixSaver (0.00s)
=== RUN   TestLookPathNotFound
--- PASS: TestLookPathNotFound (0.00s)
=== RUN   TestLookPathUnixEmptyPath
--- PASS: TestLookPathUnixEmptyPath (0.00s)
=== RUN   TestCredentialNoSetGroups
--- FAIL: TestCredentialNoSetGroups (0.00s)
        exec_posix_test.go:44: Failed to run command: fork/exec ./exec.test: operation not permitted
=== RUN   TestWaitid
=== PAUSE TestWaitid
=== RUN   TestEcho
--- PASS: TestEcho (0.00s)
=== RUN   TestCommandRelativeName
--- SKIP: TestCommandRelativeName (0.00s)
        exec_test.go:69: skipping; running test at root somehow
=== RUN   TestCatStdin
--- PASS: TestCatStdin (0.00s)
=== RUN   TestEchoFileRace
--- PASS: TestEchoFileRace (0.01s)
=== RUN   TestCatGoodAndBadFile
--- FAIL: TestCatGoodAndBadFile (0.00s)
        exec_test.go:141: expected test code; got "Error: open exec_test.go: no such file or directory\n" (len 52)
=== RUN   TestNoExistBinary
--- PASS: TestNoExistBinary (0.00s)
=== RUN   TestExitStatus
--- PASS: TestExitStatus (0.00s)
=== RUN   TestPipes
--- PASS: TestPipes (0.00s)
=== RUN   TestStdinClose
--- PASS: TestStdinClose (0.00s)
=== RUN   TestStdinCloseRace
--- PASS: TestStdinCloseRace (0.00s)
=== RUN   TestPipeLookPathLeak
--- PASS: TestPipeLookPathLeak (0.00s)
=== RUN   TestExtraFilesFDShuffle
--- SKIP: TestExtraFilesFDShuffle (0.00s)
        exec_test.go:428: flaky test; see https://golang.org/issue/5780
=== RUN   TestExtraFiles
--- PASS: TestExtraFiles (0.13s)
=== RUN   TestExtraFilesRace
--- PASS: TestExtraFilesRace (0.06s)
=== RUN   TestHelperProcess
--- PASS: TestHelperProcess (0.00s)
=== RUN   TestIgnorePipeErrorOnSuccess
=== RUN   TestIgnorePipeErrorOnSuccess/10MB
=== RUN   TestIgnorePipeErrorOnSuccess/Infinite
--- PASS: TestIgnorePipeErrorOnSuccess (0.14s)
    --- PASS: TestIgnorePipeErrorOnSuccess/10MB (0.01s)
    --- PASS: TestIgnorePipeErrorOnSuccess/Infinite (0.10s)
=== RUN   TestClosePipeOnCopyError
--- PASS: TestClosePipeOnCopyError (0.00s)
=== RUN   TestOutputStderrCapture
--- PASS: TestOutputStderrCapture (0.01s)
=== RUN   TestContext
--- PASS: TestContext (0.00s)
=== RUN   TestContextCancel
--- PASS: TestContextCancel (0.00s)
        exec_test.go:1073: exit status: signal: killed
=== RUN   TestDedupEnvEcho
--- PASS: TestDedupEnvEcho (0.00s)
=== CONT  TestWaitid
--- PASS: TestWaitid (0.20s)
FAIL

[mvdan]

If you modify the code to panic right before it re-runs itself (the test binary), and look at tmp/go-build301742174/b642/exec.test, is there anything weird about it? Perhaps the file is missing, or the permissions are wrong.

Does the issue persist if you go test -c, and run the test binary located in the src/os/exec directory?

[walac]

Indeed weird, it works in my Ubuntu Bionic vm. What does strace show?

[bcmills]

Oh, here's a clue. My current primary group is not the group that owns my go directory. (I'm still in that group, too, but it's no longer primary.)

~/go/src$ ls -alF exec.test
-rwxr-x--- 1 bcmills eng 10107712 Apr  6 18:08 exec.test*
~/go/src$ id -gn $USER
primarygroup

So the test binary is executable by group eng, but my primary group (and the GID being set in the test) is primarygroup.

[bcmills]

And indeed, after chgrp'ing my $GOROOT to the new group, it passes.

So it seems the test is sensitive to primary group changes?

[bcmills]

So, what's the right behavior here? Should the test pass even if the group doesn't match, or perhaps we should skip the test if the user's primary group doesn't match the test binary's primary group?

[ianlancetaylor]

This happened to me too. It's most likely because the LDAP server started reporting a different primary group for you; it did for me. This is not a common situation. I don't think anything needs to change in Go.