Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: one ppc64le osuosl builder is configured differently from the rest, causing failures #48614

Closed
pmur opened this issue Sep 24, 2021 · 10 comments
Labels
Builders NeedsInvestigation
Milestone

Comments

@pmur
Copy link
Contributor

@pmur pmur commented Sep 24, 2021

This is related to #48554, it would appear some power9 builders are configured differently.

Sometimes the test to open /dev/rtc0 is either skipped or passes, and sometime it fails. It is not clear if this test was always skipped previously (e.g the builder had no RTC device, or was prohibited from opening it, or has an RTC which supports alarms).

Are all the power9 builders provisioned and configured identically? Likewise, is it possible to tag golang.org/x/sys CI logs with more information about which builder they have run on (I am not sure if I am overlooking this one)?

Thanks!

@gopherbot gopherbot added the Builders label Sep 24, 2021
@gopherbot gopherbot added this to the Unreleased milestone Sep 24, 2021
@pmur
Copy link
Contributor Author

@pmur pmur commented Sep 24, 2021

@laboger fyi.

@pmur
Copy link
Contributor Author

@pmur pmur commented Sep 27, 2021

Looking into this further, it seems like we have one ppc64le power9 builder which isn't configured like the others. From farmers.golang.org I see a builder named go-ppc64le-p9 which doesn't set the HOSTNAME envvar. This matches the behavior of the failures on golang.org/x/sys.

Is this builder supposed to exist?

@laboger
Copy link
Contributor

@laboger laboger commented Sep 27, 2021

Also, this odd builder is not mentioned in x/build/env/linux-ppc64le/osuosl/NOTES and has a different distro than the others.

@laboger
Copy link
Contributor

@laboger laboger commented Sep 29, 2021

@bcmills We have found that the failures from #48554 were due to this rogue p9 builder that is not set up like the others so fails in a different way. This builder is not documented and not set up like the others. This builder should be stopped/disabled so people don't waste time tracking down bugs on this builder.

@mknyszek mknyszek added the NeedsInvestigation label Oct 4, 2021
@mknyszek mknyszek changed the title x/build: ppc64le osuosl builder configuration question x/build: one ppc64le osuosl builder is configured differently from the rest, causing failures Oct 4, 2021
@mknyszek
Copy link
Contributor

@mknyszek mknyszek commented Oct 4, 2021

CC @dmitshur @toothrot

@pmur
Copy link
Contributor Author

@pmur pmur commented Oct 4, 2021

I would like to shutdown the go-ppc64le-p9 builder. Does anyone object? Also, what steps are needed to deactivate a builder?

@cagedmantis
Copy link
Contributor

@cagedmantis cagedmantis commented Oct 4, 2021

@pmur I'm not seeing go-ppc64le-p9 on farmer.

@laboger
Copy link
Contributor

@laboger laboger commented Oct 4, 2021

I 'm not seeing go-ppc64le-p9 on farmer.

We had opened a ticket with OSU and Paul asked them to delete it. It is gone now and it looks like all is working as expected.
The builders now match what is documented in x/build/env/linux-ppc64le/osuosl/NOTES.

@cagedmantis
Copy link
Contributor

@cagedmantis cagedmantis commented Oct 4, 2021

Thank you!

@dmitshur dmitshur added this to In Progress in Go Release Team Oct 5, 2021
@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Oct 5, 2021

Closing since I understand this is all resolved, thanks again!

@dmitshur dmitshur closed this as completed Oct 5, 2021
Go Release Team automation moved this from In Progress to Done Oct 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Builders NeedsInvestigation
Projects
Development

No branches or pull requests

6 participants