x/tools/gopls: better "run this test" UX

[adonovan]

Currently, gopls offers many ways to run the test under the cursor or selected tests:

• CodeLenses, in a test.go file, returns a "run test" CodeLens (command + range) for each Test function in the entire file. The command (command.Test) runs the test with streaming output via LSP progress tokens followed by a showMessage. This feature is disabled by default, unless the client specifies because codelenses: ["test"], presumably because it is redundant with VS Code's superior client code (see below). I suspect it is almost entirely unused; telemetry would tell us for sure.

• CodeActions, in a test.go file, returns a GoTest ("Run tests and benchmarks") CodeAction for each test or benchmark in the selection. The actions commands are command.Test, as before. However, the GoTest code action kind is disabled in DefaultOptions, ~impossible to find, and again I guess that it is largely unused. Its title should be more specific (e.g. "Run TestFoo" or "Run 3 selected tests").

• VS Code Go uses VS Code's custom Test Explorer UI, and is migrating to the Testing API. See for example:

  • https://marketplace.visualstudio.com/items?itemName=ethan-reesor.vscode-go-test-adapter
  • Testing API vscode-go#1579
  • Testing API finalization microsoft/vscode#122208
  • Allow running specific test case of a test table vscode-go#2445
  • Testing API followup vscode-go#1641
  • @firelizzard18's recent work

VS Code is clearly going the path of custom client side logic, but for other editors, I think we can improve the experience.

The first challenge is to enable either the code lens or the code action, which is not trivial. Documentation may help. One possibility is to sniff the client type and enable the lens or action by default for all clients other than VS Code.

The second is the UX around the test output. Each client displays the streamed output and final showMessage in a different UI element: VS Code feeds it into a sticky dialog box; Vim accumulates the stream into a buffer which is displayed at the end. Emacs+eglot displays the output in the little echo area (bottom line), which is ephemeral and not suitable for large amounts of text.

Personally I use a custom Emacs Lisp function to fork+exec 'gopls codelens' through Emacs' compile package, so that its output is immediately streamed into a first-class editor buffer. I like this because it is immediate, real-time, editable, searchable, and durable, and because it displays the test logs even when the test passes, which is sometimes helpful. Also, Emacs treats the buffer just like a compiler output, so that locations become links, etc. The UX here is pretty close to ideal for me, but the mechanism is very clunky, and of course no-one but me benefits from it.

Yet another possibility would be to stream the output into a file and immediately open the file (using showDocument) in the client. I don't know how well different clients handle a file that is growing. (For example, Emacs has non-file buffers for things like shell sessions, which grow in real time, but for actual files, it only reloads the content periodically after polling.)

Another possibility is to stream the test output using gopls's new web server. I mention it for completeness, but it's a very loose integration and probably not a good idea.

I feel like the LSP ought to provide some way for a command to indicate to the client that its output is best handled like a terminal stream: large, continuously growing, durable, searchable. Perhaps we need to come up with a coherent plan for that and lobby our LSP overlords.