x/build: configure dashboard builders to test the same configurations that will be shipped in binary releases #46900
#43232 notes a case in which
Even if we fix that issue, we don't currently have a good way to prevent regressions, and it makes me wonder if there are other cases where we're shipping packages and/or tests that work locally but fail for users who use our provided installers.
For the platforms for which we ship official binaries, I think we should have dashboard builders that test the actual bytes that we would ship in a release — either by applying the same transformations that we will apply at release time, or by eliminating differences between the release process and
The text was updated successfully, but these errors were encountered:
TestStdlib was failing after running rm -r $(go env GOROOT)/pkg/*/cmd as the builders do when building binary releases.¹ For users who write programs that depend on go/types, it should be reasonable to run the tests for go/types as part of 'go test all', and those tests should pass even if they installed Go from a binary release. I had originally drafted this as a fallback to import from source only if the affected packages can't be imported by the default export-data importer. Unfortunately, I realized that we don't currently have a builder that tests the actual release (#46900), so it is quite likely that the fallback path would bit-rot and produce unexpected test regressions. So instead, we now unconditionally import from source in TestStdlib. That makes the test substantially slower (~15s instead of ~5s on my workstation), but with less risk of regression, and TestStdlib is skipped in short mode already so short-mode test time is unaffected. If we change the builders to test the actual release configuration, we can consider restoring the faster path when export data is available. ¹https://github.com/golang/build/blob/df58bbac082bc87c4a3cdfe336d1ffe60bbaa916/cmd/release/release.go#L533-L545 For #43232 Change-Id: I764ec56926c104053bb2ef23cf258c8f0f773290 Reviewed-on: https://go-review.googlesource.com/c/go/+/330252 Trust: Bryan C. Mills <email@example.com> Trust: Robert Griesemer <firstname.lastname@example.org> Run-TryBot: Bryan C. Mills <email@example.com> TryBot-Result: Go Bot <firstname.lastname@example.org> Reviewed-by: Robert Griesemer <email@example.com>