x/tools,x/build: performance monitoring for gopls #53538
#48803 tracks performance monitoring work in general, primarily targeting the main Go repo. x/tools folks would like to track performance for gopls, which is feasible with a bit of extra work.
The primary goal is to track the benchmarks in x/tools/gopls/internal/regtest/bench, with the latest tagged version of gopls as the baseline, a built using the latest stable Go toolchain.
Things we need to do to get there:
In the Coordinator:
This will be an x/tools builder to get proper triggering. I think we still want to use x/benchmarks/cmd/bench, so builds will have to additionally checkout x/benchmarks.
On the frontend:
 We'll still end up with a tip toolchain checked out and built on the builder which we just ignore. This is useless extra work, but I'm not sure it is worth the extra complexity to avoid.
 Perhaps we should just have cmd/bench run
The text was updated successfully, but these errors were encountered:
Regarding " Perhaps we should just have cmd/bench run go get golang.org/x/tools/gopls@latest? This is far simpler, though it diverges from how we handle the Go repo baseline (i.e., by doing it from the coordinator)" if this were outsourced to bent, autofetching at latest is the default if no version of the test repo is specified, but I don't know if the commit/date of the repo that it obtains is communicated to the database.
Significantly refactor the gopls benchmarks to turn them into proper benchmarks, eliminate the need for passing flags, and allow running them on external gopls processes so that they may be used to test older gopls versions. Doing this required decoupling the benchmarks themselves from the regtest.Runner. Instead, they just create their own regtest.Env to use for scripting operations. In order to facilitate this, I tried to redefine Env as a convenience wrapper around other primitives. By using a separate environment setup for benchmarks, I was able to eliminate a lot of regtest.Options that existed only to prevent the regtest runner from adding instrumentation that would affect benchmarking. This also helped clean up Runner.Run somewhat, though it is still too complicated. Also eliminate the unused AnyDiagnosticAtCurrentVersion, and make a few other TODOs about future cleanup. For golang/go#53992 For golang/go#53538 Change-Id: Idbf923178d4256900c3c05bc8999c0c9839a3c07 Reviewed-on: https://go-review.googlesource.com/c/tools/+/419988 gopls-CI: kokoro <email@example.com> Reviewed-by: Peter Weinberger <firstname.lastname@example.org> Run-TryBot: Robert Findley <email@example.com> TryBot-Result: Gopher Robot <firstname.lastname@example.org>
With CL 419988, gopls benchmarks are now proper benchmarks, and can be run without any additional flags. For example:
Additionally, the benchmark suite can be used with an externally compiled version of gopls:
(the bit about checking out gopls/v0.9.0 may be confusing, I realize now: that's just an arbitrary