Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: frequent "unexpected stale targets" on linux-arm-arm5spacemonkey after bootstrapping is complete #35740

Open
bcmills opened this issue Nov 21, 2019 · 11 comments

Comments

@bcmills bcmills added this to the Backlog milestone Nov 21, 2019
@bcmills
Copy link
Member Author

@bcmills bcmills commented Dec 18, 2019

This failure mode is now happening on every build.

@esnolte, you are listed as the owner of that builder here. Could you please investigate?

I wonder if the problem is somehow that multiple builds are running at the same time. (A similar failure mode in #35749 turned out to be an older builder running concurrently with the current one.)

@esnolte
Copy link

@esnolte esnolte commented Dec 18, 2019

@esnolte
Copy link

@esnolte esnolte commented Dec 18, 2019

@bcmills
Copy link
Member Author

@bcmills bcmills commented Dec 19, 2019

That doesn't seem to have fixed the problem. (https://build.golang.org/log/08d3ac2849287e18678df797c542b7329dab3d3e)

@esnolte
Copy link

@esnolte esnolte commented Dec 19, 2019

@bcmills
Copy link
Member Author

@bcmills bcmills commented Dec 19, 2019

Hmm, now https://farmer.golang.org/ shows 0/2 host-linux-arm5spacemonkey hosts available, but lists them both as connected:

Reverse pool by host type (in use / total)

  • […]
  • host-linux-arm5spacemonkey: 0/2 (1 missing)
  • go-builder-1 (69.58.94.166:38292) version 24, host-linux-arm5spacemonkey: connected 50m45.9s, idle for 3.41s
  • go-builder-2 (69.58.94.166:27320) version 24, host-linux-arm5spacemonkey: connected 1h18m43.9s, idle for 10.2s
  • linux-arm-arm5spacemonkey rev a197c7b; waiting_for_machine, 2m28s ago
@dmitshur
Copy link
Member

@dmitshur dmitshur commented Dec 19, 2019

I've unplugged a rogue builder we no longer have access to (go-builder-3). It looks like the other 2 builders are passing tests.

By now, this seems to be accurately reflected under the "Reverse pool by host type (in use / total)" section of https://farmer.golang.org/#pools:

host-linux-arm5spacemonkey: 2/2 (1 missing)

I'm not sure why it was showing something else before.

@esnolte
Copy link

@esnolte esnolte commented Dec 19, 2019

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Dec 23, 2019

The original problem is still happening.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Dec 27, 2019

IIRC, the arm5 builders start with a cross-compiled make.bash from a linux-amd64 machine. Perhaps the two environments have drifted?

@esnolte
Copy link

@esnolte esnolte commented Jan 21, 2020

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.