runtime: builders failing on macOS 10.11 darwin/amd64 and flaky on other darwin releases

[dmitshur]

As of CL 182258, the macOS 10.11 darwin/amd64 builders are failing on https://build.golang.org.

/cc @randall77 @ianlancetaylor

[katiehockman]

@dmitshur Should we roll back the original change in the meantime so the builders start working?

[dmitshur]

In this case, I don't think rolling back will put us in a better state. The CL has fixed another macOS issue (#31264) that was causing macOS 10.14 and nocgo builders to fail very often:

Rolling it back would fix 10.11 builder but break two other builders. I suspect it's better to try to fix the issue forward.

[eliasnaur]

It seems that similar failures can happen on the nocgo darwin builder (10.14?), albeit at a lower rate than on 10.11. For example:

https://build.golang.org/log/038830c3ffccb9a97610a16f5d8653644dcb41ea

Regarding SIGILL, I came across this section in Apple's documentation:

Important
Calls to dispatch_semaphore_signal must be balanced with calls to wait(). Attempting to dispose of a semaphore with a count lower than value causes an EXC_BAD_INSTRUCTION exception.

I'm not familiar with the code so I can't say whether that note is relevant.

[ianlancetaylor]

I ran all.bash on the Darwin 10.11 gomote, and it passed. But the builder is failing consistently on the os/signal test. I'm not sure why.

[ianlancetaylor]

I don't know what the problem is. The note from Apple's docs doesn't tell me anything, as we never dispose of these semaphores. Given that the problem can happen on 10.14, I think we should roll back CL 182258. It's possible that the new code is not async-signal-safe, just as the old code was not. And the old code fails less often.

[gopherbot]

Change https://golang.org/cl/182880 mentions this issue: Revert "runtime: use dispatch semaphores on Darwin"

[ianlancetaylor]

Sent CL 182880 to revert the earlier CL 182258. That will fix this issue but force us to reopen #31264.