Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: high rate of i/o timeout errors on linux-s390x-ibm builder since 2022-03-31 #52240

Closed
bcmills opened this issue Apr 8, 2022 · 4 comments
Closed
Labels
arch-s390x Builders NeedsInvestigation
Milestone

Comments

@bcmills bcmills added NeedsInvestigation arch-s390x labels Apr 8, 2022
@gopherbot gopherbot added the Builders label Apr 8, 2022
@gopherbot gopherbot added this to the Unreleased milestone Apr 8, 2022
@bcmills bcmills removed this from the Unreleased milestone Apr 8, 2022
@bcmills bcmills added this to the Backlog milestone Apr 8, 2022
@jonathan-albrecht-ibm
Copy link
Contributor

@jonathan-albrecht-ibm jonathan-albrecht-ibm commented Apr 8, 2022

@bcmills I'll have a look

@jonathan-albrecht-ibm
Copy link
Contributor

@jonathan-albrecht-ibm jonathan-albrecht-ibm commented Apr 8, 2022

The servers seem to be timing out on the ipv6 address for proxy.golang.org. I don't know if its timing out for every ipv6 address. I've changed the network config to prefer ipv4 addresses first and it seems fast for proxy.golang.org now.

I'll watch to see if the i/o timeout errors happen again over the weekend.

@bcmills I tried running fetchlogs on the repos in the failed logs in the description but I don't get all the failures you do when I run greplogs . Do you have a set of fetchlogs command lines you could give me? I want to make sure I'm not missing any.

@bcmills
Copy link
Member Author

@bcmills bcmills commented Apr 12, 2022

I tried running fetchlogs on the repos in the failed logs in the description but I don't get all the failures you do when I run greplogs.

I typically run fetchlogs -repo all -n 1024 to pull in the recent failures. You may also need a greplogs with some recent fixes (see aclements/go-misc#10).

@jonathan-albrecht-ibm
Copy link
Contributor

@jonathan-albrecht-ibm jonathan-albrecht-ibm commented Apr 19, 2022

I'm building cmd/fetchlogs from https://go.googlesource.com/build but it doesn't recognize all as an value for -repo so I'm just looping over the known ones in a script. I haven't seen any new failures of this type since I changed the network config.

@bcmills, I think it should be okay to close this if it looks ok to you too.

@bcmills bcmills closed this as completed Apr 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arch-s390x Builders NeedsInvestigation
Projects
None yet
Development

No branches or pull requests

3 participants