Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: builders failing on macOS 10.11 darwin/amd64 and flaky on other darwin releases #32655

Closed
dmitshur opened this issue Jun 17, 2019 · 7 comments

Comments

@dmitshur
Copy link
Member

@dmitshur dmitshur commented Jun 17, 2019

As of CL 182258, the macOS 10.11 darwin/amd64 builders are failing on https://build.golang.org.

/cc @randall77 @ianlancetaylor

@katiehockman

This comment has been minimized.

Copy link
Member

@katiehockman katiehockman commented Jun 18, 2019

@dmitshur Should we roll back the original change in the meantime so the builders start working?

@dmitshur

This comment has been minimized.

Copy link
Member Author

@dmitshur dmitshur commented Jun 18, 2019

In this case, I don't think rolling back will put us in a better state. The CL has fixed another macOS issue (#31264) that was causing macOS 10.14 and nocgo builders to fail very often:

Rolling it back would fix 10.11 builder but break two other builders. I suspect it's better to try to fix the issue forward.

@eliasnaur

This comment has been minimized.

Copy link
Contributor

@eliasnaur eliasnaur commented Jun 18, 2019

It seems that similar failures can happen on the nocgo darwin builder (10.14?), albeit at a lower rate than on 10.11. For example:

https://build.golang.org/log/038830c3ffccb9a97610a16f5d8653644dcb41ea

Regarding SIGILL, I came across this section in Apple's documentation:

Important
Calls to dispatch_semaphore_signal must be balanced with calls to wait(). Attempting to dispose of a semaphore with a count lower than value causes an EXC_BAD_INSTRUCTION exception.

I'm not familiar with the code so I can't say whether that note is relevant.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 18, 2019

I ran all.bash on the Darwin 10.11 gomote, and it passed. But the builder is failing consistently on the os/signal test. I'm not sure why.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 18, 2019

I don't know what the problem is. The note from Apple's docs doesn't tell me anything, as we never dispose of these semaphores. Given that the problem can happen on 10.14, I think we should roll back CL 182258. It's possible that the new code is not async-signal-safe, just as the old code was not. And the old code fails less often.

@bcmills bcmills changed the title runtime: builders failing on macOS 10.11 darwin/amd64 runtime: builders failing on macOS 10.11 darwin/amd64 and flaky on other darwin releases Jun 19, 2019
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Jun 19, 2019

Change https://golang.org/cl/182880 mentions this issue: Revert "runtime: use dispatch semaphores on Darwin"

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 19, 2019

Sent CL 182880 to revert the earlier CL 182258. That will fix this issue but force us to reopen #31264.

@gopherbot gopherbot closed this in 1130cc7 Jun 21, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.