x/build: automate testing "toolstash -cmp" with trybots

[mdempsky]

For cleanup CLs like https://go-review.googlesource.com/#/c/8762, it would be nice if the trybots could take care of building a toolchain at HEAD, toolstash saving it, rebuilding at HEAD+CL, and then running an appropriate "toolstash -cmp" build.

Using toolstash/buildall, this only needs to be run once on a fast/cheap trybot type (e.g., linux/amd64) to avoid wasting more expensive trybot resources (e.g., OS X and/or ARM).

Obviously this would only be on an opt-in basis. Perhaps just a new dedicated trybot type that doesn't run by default, is configured with a different testing script than running all.bash, and is selected via whatever +trybot mechanism is implemented.

Maybe too niche a use case, but some rationale:

• Now that the toolchain is in Go, it seems like cleanup CLs to make the code more idiomatic are going to be increasingly common.
• It's easy to misuse toolstash; e.g., accidentally build the stashed toolchain at HEAD+CL rather than HEAD, negating the testing benefits.
• As a reviewer, you need to trust that developers who say "Passes toolstash -cmp" actually tested it correctly.

[josharian]

I'd love this.

[mdempsky]

Tangentially, I just realized that using "toolstash -cmp" with "go build std" doesn't provide any linker coverage. It should be something like "go build std cmd".

[gopherbot]

CL https://golang.org/cl/9530 mentions this issue.

[bradfitz]

Probably the easiest way to implement this is to take advantage of the existing post-make.bash snapshots we already upload to GCS for all builds.

So after trybots pass (probably for each build config? or at least Windows + Linux?), pull down tarballs of the git parent's previous make.bash snapshot and the current one, and toolstash cmp that, or equivalent.

/cc @toothrot