Proposal Details
I propose adding a Location field to JSON "test started" events specifically for subtests. This field would show the full file path and line of the t.Run call.
As a contributor to vscode-go and other tooling, I want to provide a complete experience to users (to Go developers using these tools). Specifically, I want vscode-go's test explorer (the implementation of VSCode's testing API) to provide a complete experience. For statically defined tests (func TestFoo(t *testing.T)), it is easy to determine the name and location of the test, as was pointed out in #45733. For dynamically defined tests (calls to (*testing.T).Run), it is not. Detecting a subset of calls via static analysis is possible, and I contributed that to gopls, but complete detection is impossible (the halting problem) and even 80-90% coverage is technically infeasible (it would impose an unacceptably great engineering burden on gopls). Currently, while tooling can report that a subtest exists (because there will be a "test started" JSON event), tooling cannot determine the location of that subtest (unless it can be statically determined, which has the limitations mentioned above).
Hence, I propose adding a Location field to go test -json "test started" events. When t.Run is called, the caller's PC is already captured, so there are only two necessary changes: the PC needs to be converted to a file:line and emitted, and test2json needs to capture that and attach it to the "test started" event. The options I see are:
- Emit
=== RUN {name} @ {location} when t.chatty != nil (so, -v or -json) or when t.chatty.json is true (only for -json).
- We could use some marker to wrap the location (instead of `{name} @ {location}, but that seems needlessly complicated, since test names (as emitted) can't contain spaces.
- Emit
=== LOC {name} {location} before RUN.
- Emit
=== ATTR {name} {special attr key} {location} before RUN.
- This could conflict with user generated attributes so personally I prefer LOC.
These are all functionally equivalent in terms of implementation difficulty and expressiveness. Extended-RUN might have some potential parsing nuances, but given the constraints on test names (spaces are converted to underscores, etc), I think that's a non-issue.
Emitting LOC or ATTR after run is problematic, because test2json emits the "test started" event immediately upon seeing RUN, so waiting for LOC or ATTR after RUN would require some kind of buffering (which IMO would be needlessly complicated).
If this is accepted, I am happy to submit the CL myself. Due to my prior experience with CL 751940 and CL 601535 I think it would be relatively simple.
Proposal Details
I propose adding a
Locationfield to JSON "test started" events specifically for subtests. This field would show the full file path and line of thet.Runcall.As a contributor to vscode-go and other tooling, I want to provide a complete experience to users (to Go developers using these tools). Specifically, I want vscode-go's test explorer (the implementation of VSCode's testing API) to provide a complete experience. For statically defined tests (
func TestFoo(t *testing.T)), it is easy to determine the name and location of the test, as was pointed out in #45733. For dynamically defined tests (calls to(*testing.T).Run), it is not. Detecting a subset of calls via static analysis is possible, and I contributed that to gopls, but complete detection is impossible (the halting problem) and even 80-90% coverage is technically infeasible (it would impose an unacceptably great engineering burden on gopls). Currently, while tooling can report that a subtest exists (because there will be a "test started" JSON event), tooling cannot determine the location of that subtest (unless it can be statically determined, which has the limitations mentioned above).Hence, I propose adding a
Locationfield togo test -json"test started" events. Whent.Runis called, the caller's PC is already captured, so there are only two necessary changes: the PC needs to be converted to a file:line and emitted, and test2json needs to capture that and attach it to the "test started" event. The options I see are:=== RUN {name} @ {location}whent.chatty != nil(so, -v or -json) or whent.chatty.jsonis true (only for -json).=== LOC {name} {location}before RUN.=== ATTR {name} {special attr key} {location}before RUN.These are all functionally equivalent in terms of implementation difficulty and expressiveness. Extended-RUN might have some potential parsing nuances, but given the constraints on test names (spaces are converted to underscores, etc), I think that's a non-issue.
Emitting LOC or ATTR after run is problematic, because test2json emits the "test started" event immediately upon seeing RUN, so waiting for LOC or ATTR after RUN would require some kind of buffering (which IMO would be needlessly complicated).
If this is accepted, I am happy to submit the CL myself. Due to my prior experience with CL 751940 and CL 601535 I think it would be relatively simple.