Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: "unexpected stale targets" on plan9-arm #49691

Open
bcmills opened this issue Nov 19, 2021 · 6 comments
Open

x/build: "unexpected stale targets" on plan9-arm #49691

bcmills opened this issue Nov 19, 2021 · 6 comments

Comments

@millerresearch
Copy link

@millerresearch millerresearch commented Nov 19, 2021

The error message is not that helpful for the uninitiated. What exactly does "build ID mismatch" mean? Is the staleness algorithm documented somewhere?

It can sometimes happen that make.rc is run on one builder, the result is saved as a snapshot and used to run 'dist test' on another builder. If the two builders are not configured exactly the same, is that likely to result in stale targets? I would hope not, because the snapshot contains all the source, object code, and toolchain, so it should be self-consistent in any environment. But it's hard to be certain without a definition of what "stale" is supposed to mean.

In some of the logs, there is clearly some other filesystem-related failure first (some bits of the tree have disappeared), which is likely what triggers the stale target report. But in other examples, the stale target report seems to be the first or only thing that has gone wrong.

Loading

@bcmills
Copy link
Member Author

@bcmills bcmills commented Nov 19, 2021

Loading

@bcmills
Copy link
Member Author

@bcmills bcmills commented Nov 19, 2021

The hash inputs include any relevant flags, tool versions, and whatever directory information will end up in the binary (which is itself a function of GOROOT_FINAL, GOROOT, and the working directory for the build.)

Loading

@bcmills
Copy link
Member Author

@bcmills bcmills commented Nov 19, 2021

In the latest failure I see this line:

HASH[link cmd/dist]: "GOROOT=/tmp/workdir-rpi31/go\n"

Do all of the builders end up with the same working directory (/tmp/workdir-rpi31)? If not, then that could be the cause.

In #33598 we found that the darwin builders were sensitive to the PWD environment variable: their GOROOT was in a symlinked directory, so without a correct PWD they ended up reporting the underlying directory rather than the original path. Is there a similar consideration for Plan 9?

Loading

@millerresearch
Copy link

@millerresearch millerresearch commented Nov 21, 2021

There are two sets of plan9-arm Raspberry Pi builders, run by @0intro and myself, running slightly different scripts. Mine all use /boot/workdir/go as GOROOT, but David's use a different directory on each host because they're sharing the same server filesystem. @0intro, may I suggest that you add a bind in your script to use the same GOROOT as mine, and that should prevent stale target failures when the build and tests are run on different hosts.

That won't fix all cases, though. For example in this one, the tests are running immediately after building the whole system on the same machine, and getting stale dependencies on vendor/golang.org/x/text/unicode/norm. How is that possible? Is there something tricky about how the vendor subtree is built?

Loading

@0intro
Copy link
Member

@0intro 0intro commented Nov 21, 2021

@0intro, may I suggest that you add a bind in your script to use the same GOROOT as mine, and that should prevent stale target failures when the build and tests are run on different hosts.

Yes, I'll try that.

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants