x/build/cmd/coordinator: include -longtest and -race builders for trybots on release branches #37827
After a major version of a Go release is out, we send relatively few CLs to the release branch in order to backport fixes for minor releases, per https://golang.org/wiki/MinorReleases policy.
We should consider including -longtest and -race builders in the default tryset (set of builders included in a normal trybot run) for those branches::
Those builders already run as post-submit builders, but also running them during trybots will give us more information to work with before a CL is submitted to a release branch.
We may want to do this only for minor releases, i.e., after a major Go release is out. Or we can do this always, which would include CLs sent to a release branch after the first release candidate is made.
An existing alternative is to manually request those builders via SlowBots.
The text was updated successfully, but these errors were encountered:
This test data will be modified to match a change in behavior in the next commit. Update it to a supported Go version first. For golang/go#37827. Change-Id: I83ad967a11f0fdde454edc541e9a8e7cf061f0e0 Reviewed-on: https://go-review.googlesource.com/c/build/+/235337 Run-TryBot: Dmitri Shuralyov <firstname.lastname@example.org> TryBot-Result: Gobot Gobot <email@example.com> Reviewed-by: Carlos Amedee <firstname.lastname@example.org> Reviewed-by: Alexander Rakoczy <email@example.com>
The goal of this test policy change is to decrease the chance of a Go repository release branch from getting into a state where some longtest post-submit builders are failing, as this will interfere with the ability to issue releases after golang/go#29252 is fixed. There are not that many CLs sent to release branches, so the extra latency caused by slower longtest builders being a part of trybots should not be very disruptive. For golang/go#37827. For golang/go#29252. Change-Id: I819b120bb7b763df4acf4583375cdb840793d323 Reviewed-on: https://go-review.googlesource.com/c/build/+/235338 Run-TryBot: Dmitri Shuralyov <firstname.lastname@example.org> TryBot-Result: Gobot Gobot <email@example.com> Reviewed-by: Alexander Rakoczy <firstname.lastname@example.org> Reviewed-by: Carlos Amedee <email@example.com>
CL 235338 is deployed now.
I tried it in an empty CL on release-branch.go1.13 (see here) and the amd64 longtest builders are working as expected, but linux-386-longtest is failing in a strange way:
The goal of this change is to reduce the chance of issuing a release with an unintended regression that would be caught by running long tests. This change adds long tests that are run during the -prepare step, in addition to all the existing short tests that are run. Executing the long tests is implemented by adding two new test-only release targets. For a release to be considered complete, all release targets must complete. These test-only targets are built only for the purpose of verifying their tests succeed, or to block the release otherwise. They do not produce release artifacts. The new test-only targets are named after the builder which is used to perform their tests, and they are: • linux-amd64-longtest • windows-amd64-longtest More builders may be added in the future, but care must be taken to ensure the test execution environment is as close as possible to that of build.golang.org post-submit builders, in order to avoid false positives and false negatives. As part of a gradual rollout, this change also adds a flag to skip longtest builders. It's meant to be used in case a long test proves to be flaky, and enough confidence can be gained through testing elsewhere that the failure is not a regression caused by a change merged to the release branch. For now, its default value includes both longtest builders, so they are currently opt-in and this CL is a no-op. After testing proves that it is viable to rely on this (and any issues preventing that from being possible are resolved), the default value of the flag will be changed to the empty string. For golang/go#29252. For golang/go#39054. For golang/go#37827. Fixes golang/go#24614. Change-Id: I33ea6601aade2873f857c63f00a0c11821f35a95 Reviewed-on: https://go-review.googlesource.com/c/build/+/234531 Run-TryBot: Dmitri Shuralyov <firstname.lastname@example.org> TryBot-Result: Gobot Gobot <email@example.com> Reviewed-by: Carlos Amedee <firstname.lastname@example.org> Reviewed-by: Alexander Rakoczy <email@example.com>