Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
proposal: cmd/go: breakout testmain generation functionality into its own tool #30421
When go test is ran there is logic in
External build tools like Bazel (https://bazel.build/) and Buck (https://buckbuild.com/) bypass the go tool and execute the compiler and linkers directly to get control over the output archives, etc to use for distributed caching and to unify building across multiple languages. To deal with tests both Buck and Bazel have done a copy and paste of the testmain generation code with some modifications and embedded them.
Being an internal package, obviously it is not a stable API. Every Go version there can be minor changes to how tests are discovered or how the testmain code generation is code. These have to be painstakingly backported into Bazel and Buck to keep working while also maintaining comparability with older Go versions.
Since the generated testmain code is linked with the testing library found in the stdlib it would make sense that there is a clear way to generate testmain code that is tied to the version of Go toolchain.
I am proposing abstracting out the test discovery and testmain generation code into it's own external tool (e.g go tool testmaingen) that can be called from external tools. The go tool could call either a shared library or the testmaingen tool directly.
This would be similar to how cgo code generation is done.
You can kind of already do this. If you run
Something that complicates this change is that we generate the testmain source when loading packages, not during execution as you'd expect. The generated testmain is listed in
Even if we did this, I'd be very surprised if the generated file could be used as-is. Bazel needs to do some other stuff in
That is pretty interesting. I didn't know it was stored in the cache. Digging through the cache seems like it could be more fragile than what we are dealing with here.
In Buck we change directories , do test discovery, do sharding through a "test runner" which is just a wrapper around the compiled testmain which seems like a much cleaner solution with better abstractions, but that isn't really the topic here.
Does Bazel/Blaze do this for other languages as well like Python or C++?
There is very little that Buck does here intentionally as it's already too hard to maintain the testmain generation code with just changes to the
The core problem we have had w/ Buck is that the testing package and the generated code that uses has changed with nearly every major Go release. We have been able to hack around this by adding reflection, implementing an interface, etc Even if some of the testmaingen code was available as an external library that could be imported it would help, but it would require a stable API between it and the
Here are some of the changes that has been required.
In fact when we upgrade to new Go versions this is really the only consistent blocker we have had over the years.
I think @jayconrod's answer here is the right one. There shouldn't be a separate API for generating testmain.go by itself. Even knowing that testmain.go is a thing is too much detailed knowledge about tests. What we've exposed is how to ask 'tell me the exact way the test binary should get built' instead. That's a much more general query.