Test does not run correctly if subtest name contains regexp metacharacters

[mattwelke]

What version of Go, VS Code & VS Code Go extension are you using?

Version Information

• Run go version to get version of Go from the VS Code integrated terminal.
  • go version go1.21.3 linux/amd64
• Run gopls -v version to get version of Gopls from the VS Code integrated terminal.
  • v0.14.2
• Run code -v or code-insiders -v to get version of VS Code or VS Code Insiders.
  • 1.84.2
• Check your installed extensions to get the version of the VS Code Go extension
  • v0.40.0
• Run Ctrl+Shift+P (Cmd+Shift+P on Mac OS) > Go: Locate Configured Go Tools command.
  • Output:

Checking configured tools....
GOBIN: undefined
toolsGopath: 
gopath: /home/matt/go
GOROOT: /usr/local/go
PATH: /home/matt/.pyenv/shims:/home/matt/.pyenv/bin:/home/matt/.rbenv/shims:/home/matt/.rbenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/home/matt/.local/share/JetBrains/Toolbox/scripts:/usr/local/go/bin:/home/matt/cpu-temp-script:/home/matt/go/bin:/usr/local/bin/

	go:	/usr/local/go/bin/go: go version go1.21.3 linux/amd64

	gopls:	/home/matt/go/bin/gopls	(version: v0.14.2 built with go: go1.21.3)
	gotests:	/home/matt/go/bin/gotests	(version: v1.6.0 built with go: go1.21.3)
	gomodifytags:	/home/matt/go/bin/gomodifytags	(version: v1.16.0 built with go: go1.21.3)
	impl:	/home/matt/go/bin/impl	(version: v1.1.0 built with go: go1.21.3)
	goplay:	/home/matt/go/bin/goplay	(version: v1.0.0 built with go: go1.21.3)
	dlv:	/home/matt/go/bin/dlv	(version: v1.21.2 built with go: go1.21.3)
	staticcheck:	/home/matt/go/bin/staticcheck	(version: v0.4.6 built with go: go1.21.3)

go env
Workspace Folder (golang-sandbox): /home/matt/golang-sandbox
	GO111MODULE=''
	GOARCH='amd64'
	GOBIN=''
	GOCACHE='/home/matt/.cache/go-build'
	GOENV='/home/matt/.config/go/env'
	GOEXE=''
	GOEXPERIMENT=''
	GOFLAGS=''
	GOHOSTARCH='amd64'
	GOHOSTOS='linux'
	GOINSECURE=''
	GOMODCACHE='/home/matt/go/pkg/mod'
	GONOPROXY=''
	GONOSUMDB=''
	GOOS='linux'
	GOPATH='/home/matt/go'
	GOPRIVATE=''
	GOPROXY='https://proxy.golang.org,direct'
	GOROOT='/usr/local/go'
	GOSUMDB='sum.golang.org'
	GOTMPDIR=''
	GOTOOLCHAIN='auto'
	GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
	GOVCS=''
	GOVERSION='go1.21.3'
	GCCGO='gccgo'
	GOAMD64='v1'
	AR='ar'
	CC='gcc'
	CXX='g++'
	CGO_ENABLED='1'
	GOMOD='/home/matt/golang-sandbox/go.mod'
	GOWORK=''
	CGO_CFLAGS='-O2 -g'
	CGO_CPPFLAGS=''
	CGO_CXXFLAGS='-O2 -g'
	CGO_FFLAGS='-O2 -g'
	CGO_LDFLAGS='-O2 -g'
	PKG_CONFIG='pkg-config'
	GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2574103532=/tmp/go-build -gno-record-gcc-switches'

Share the Go related settings you have added/edited

Run Preferences: Open Settings (JSON) command to open your settings.json file.
Share all the settings with the go. or ["go"] or gopls prefixes.

"[go]": {
    "editor.wordWrapColumn": 100,
    "editor.rulers": [100, 120]
},

Describe the bug

I expected that when clicking on "Debug Test" for a subtest in the test explorer, with a valid breakpoint set, my breakpoint would be hit. Instead, my breakpoint isn't hit, and I get a message in the debug console telling me that there are "no tests to run".

Steps to reproduce the behavior:

Complete these steps as I did earlier...

I created a function and a test table for it using Go: Generate Unit Tests For Function from the command palette. I filled in the test table with one test that I expected to pass and one that I expected to fail. My code:

package main

import "fmt"

func sum(a, b int) int {
	return 2
}

func main() {
	a := 1
	b := 2
	fmt.Printf("The sum of %d and %d is %d.\n", a, b, sum(a, b))
}

package main

import "testing"

func Test_sum(t *testing.T) {
	type args struct {
		a int
		b int
	}
	tests := []struct {
		name string
		args args
		want int
	}{
		{
			name: "1_+_1_=_2",
			args: args{
				a: 1,
				b: 1,
			},
			want: 2,
		},
		{
			name: "1 + 2 = 3",
			args: args{
				a: 1,
				b: 2,
			},
			want: 3,
		},
	}
	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			if got := sum(tt.args.a, tt.args.b); got != tt.want {
				t.Errorf("sum() = %v, want %v", got, tt.want)
			}
		})
	}
}

I opened the test explorer tab:

I clicked "Run Test" on the root test (where my module name is). It then ran my tests:

I added a break point to my test body:

I clicked "Debug Test" just for that subtest/test table entry:

My breakpoint was not hit. The process finished and the debug console displayed:

Starting: /home/matt/go/bin/dlv dap --listen=127.0.0.1:35709 --log-dest=3 from /home/matt/golang-sandbox
DAP server listening at: 127.0.0.1:35709
Type 'dlv help' for list of commands.
testing: warning: no tests to run
PASS
Process 108182 has exited with status 0
Detaching

I clicked "Debug Test" at the test level instead of the subtest level:

This time, my breakpoint was hit, presumably because all of the subtests are being run now, in debug mode:

[hyangah]

$ go test
--- FAIL: Test_sum (0.00s)
    --- FAIL: Test_sum/1_+_2_=_3 (0.00s)
        main_test.go:35: sum() = 2, want 3
FAIL
exit status 1
FAIL    example/test    0.136s
$ go test -run=^Test_sum/1_+_2_=_3$
testing: warning: no tests to run
PASS
ok      example/test    0.131s

This is due to the use of + which is a regexp metacharacter. The extension has to scrub the discovered test name further to escape these.

[mattwelke]

Ah interesting. I ran into this issue with my actual test suite and when putting this minimal reproduction together for this report, I was able to reproduce it because I happened to also choose a character that was a regexp metacharacter. You can see I even tested whether the test name was related to the issue by changing my test name in this minimal reproduction from 1 + 1 = 2 to 1_+_1_=_2 (getting rid of spaces).

I don't recall what the test names were in my actual codebase, but if it would help with solving this issue I can go back to it and try this again, and report back.

[adonovan]

I fixed this (by chance this morning!) in https://go.dev/cl/546419 (see gopls/internal/server/command.go), when I noticed the code looked wrong. I confirmed the fix using VS Code and the gopls binary before/after this commit.

Run go install golang.org/x/tools/gopls@latest to try out the fix.

[hyangah]

Thanks Alan. But VS Code Go extension does not use gopls for its test and debug execution functionality.
The relevant code location is, I think,
https://github.com/golang/vscode-go/blob/322014f62edb664358415885cd0dc56a23249aaa/src/subTestUtils.ts#L21C19-L21C19

p.s does gopls test codelens already support subtests? That will be great.

EDIT: the relevant code location is more deep in the test explorer run.ts. Keep looking...

[adonovan]

Thanks Alan. But VS Code Go extension does not use gopls for its test and debug execution functionality.

Funny, that's what I thought, but I tested the old and new gopls binaries, ABAB, and got consistent results. How strange.

p.s does gopls test codelens already support subtests? That will be great.

The codelens only lists top-level functions. But the the executeCommand(gopls.test) operation accepts any test name, including subtests, and passes it straight through to go test. I assumed that VS Code was using its native logic to figure out the names of all the subtests, but executeCommand to re-run a selected subtest.

[findleyr]

I assumed that VS Code was using its native logic to figure out the names of all the subtests, but executeCommand to re-run a selected subtest.

VS Code uses completely separate logic and implementation.
At one point, there was thought of it using middleware to intercept the gopls code actions to identify subtests, and then pipe them into the native runner (so, the opposite of how you guessed it worked...).

[adonovan]

VS Code uses completely separate logic and implementation. At one point, there was thought of it using middleware to intercept the gopls code actions to identify subtests, and then pipe them into the native runner (so, the opposite of how you guessed it worked...).

Thanks for clarifying. I just checked again and now I can't reproduce what I saw before. Sorry for the confusion.

[gopherbot]

Change https://go.dev/cl/546260 mentions this issue: src/goTest: fix bugs in subtest handling