x/build: well-lit path for testing CLs with platform dependencies

[neild]

Background: https://go.dev/cl/565255 introduced failures on a variety of ports (#65906, #65907), despite passing initial LUCI checks.

This is a CL that involves GOOS/GOARCH-specific networking behavior, so it isn't surprising the default LUCI configuration set wasn't sufficient to adequately test it.

There should be a well-lit path for testing weird CLs like this one. You can manually add additional configurations to LUCI, but this involves a lot of clicking around in the Gerrit UI and you need to manually decide which configurations to add. We don't need a single "test this harder" button (although it would be nice), but there should at least be a documented set of steps to take when presubmit-testing a CL that's more likely than the norm to have failure modes that don't show up in the default configurations.

[mknyszek]

In terms of a 'test this harder' button, you could imagine a special builder called "x_net-full" or something, which is really just a proxy builder. It understands the set of actual builders in postsubmit that work, and that aren't already in the default presubmit set. It launches builds for them and waits for them to complete. The downstream builds it launches are also able to appear in the Gerrit UI independently.

I don't think this would be too hard to implement, but it is still additional work.

[bcmills]

See also:

• x/build: adding LUCI builders blocking submit is too manual #65439

[bcmills]

For this particular use-case, I wonder if it would suffice to filter out all of the unsupported builders (is there a tracking issue for that?) along with a “select all” button.

[dmitshur]

CC @golang/release.

[mknyszek]

@bcmills That would work too, though because the presubmit set isn't sticky unless you modify your commit message, the list is going to be very long. Not really a technical problem, but annoying for a human to read. There's a tracking issue for the stickiness (though it requests it in a bit of an indirect manner), which is https://issues.chromium.org/issues/40287467.

There is no tracking issue for that. I'll file one. I recently looked into exactly what it would take to shrink the builder set by merging builder definitions, so that all builders listed are always applicable.

Separately, I also should have a tracking issue for filtering out known issue builders. That is easy to do and we should just do it. What's left after that are just inapplicable builders (like go1.22-linux-amd64 for a CL that's against the main branch).

[prattmic]

filtering out known issue builders

I'm not convinced about this. If my CL is supposed to fix a known issue builder, then how do I test it?

[mknyszek]

'Doh. I see your point. Clearly I did not think that through.