Support running single tests that are not inside their group hierarchy in the source file #3630

jvretamero · 2021-10-26T04:40:22Z

Is your feature request related to a problem? Please describe.
Having a group with an anonymous function with some calls to the test function, when I click on "Run" above an test function, somehow the argument passed to dart test has the group an test name and I'm able to run the test without problems.

But when I pass a reference to a function to the group, the argument passed to dart test has only the test name and I receive an No tests match regular expression error.

Describe the solution you'd like
This is more a question than an enchancement, so here it goes: Is it possible to run the First test in the example by clicking on "Run" above it?

I'm questioning because I don't know if there is a limitation on how VS Code handle this.

Additional context
Here is a sample code. In this example, I'm able to run "Second test" but not "First test".

import 'package:test/test.dart';

void main() {
  group('First group', referencedFunction);
  group('Second group', () {
    test('Second test', () {});
  });
}

void referencedFunction() {
  test('First test', () {});
}

And also the debug logs of both scenarios.

!! PLEASE REVIEW THIS LOG FOR SENSITIVE INFORMATION BEFORE SHARING !!

Dart Code extension: 3.27.2
Flutter extension: 3.27.0 (not activated)

App: Visual Studio Code
Version: 1.61.2
Platform: win

Workspace type: Dart
Analyzer type: LSP
Multi-root?: false

Dart SDK:
    Loc: C:\flutter-2.5.3\bin\cache\dart-sdk
    Ver: 2.14.4
Flutter SDK:
    Loc: C:\flutter-2.5.3
    Ver: 2.5.3

HTTP_PROXY: undefined
NO_PROXY: undefined

Logging Categories:
    General, CommandProcesses, DAP, VmService, DevTools, FlutterDaemon, FlutterRun, FlutterTest, PubTest, WebDaemon

Tue Oct 26 2021 [01:32:19 GMT-0300 (Horário Padrão de Brasília)] Log file started
[01:32:21] [General] [Info] Starting debug session...
[01:32:21] [General] [Info]     workspace: C:\Temp\test_test
[01:32:21] [General] [Info]     program  : C:\Temp\test_test\test\simple_test.dart
[01:32:21] [General] [Info] Using workspace as cwd: C:\Temp\test_test
[01:32:21] [General] [Info] Project (C:\Temp\test_test\test\simple_test.dart) not recognised as Flutter or Web, will use Dart debugger
[01:32:21] [General] [Info] Detected launch project as Dart
[01:32:21] [General] [Info] Detected launch project as a Test project
[01:32:21] [General] [Info] Using PubTest debug adapter for this session
[01:32:21] [General] [Info] Checking for errors before launching
[01:32:21] [General] [Info] Debug session starting...
    {
        "suppressPromptOnErrors": false,
        "name": "Tests simple_test.dart",
        "noDebug": true,
        "request": "launch",
        "type": "dart",
        "expectSingleTest": true,
        "program": "C:\\Temp\\test_test\\test\\simple_test.dart",
        "toolArgs": [
            "--name",
            "^First test( \\(variant: .*\\))?$",
            "--run-skipped"
        ],
        "dartCodeDebugSessionID": "session-2e96",
        "cwd": "C:\\Temp\\test_test",
        "toolEnv": {
            "FLUTTER_HOST": "VSCode",
            "PUB_ENVIRONMENT": "vscode.dart-code"
        },
        "sendLogsToClient": true,
        "additionalProjectPaths": [
            "C:\\Temp\\test_test"
        ],
        "args": [],
        "vmServicePort": 0,
        "dartSdkPath": "C:\\flutter-2.5.3\\bin\\cache\\dart-sdk",
        "maxLogLineLength": 2000,
        "debugSdkLibraries": false,
        "debugExternalPackageLibraries": false,
        "showDartDeveloperLogs": true,
        "evaluateGettersInDebugViews": true,
        "evaluateToStringInDebugViews": true,
        "debuggerType": 1
    }
[01:32:21] [General] [Info] Using dart_test debugger for PubTest
[01:32:21] [General] [Info] Running debugger via node with c:\.vscode\extensions\dart-code.dart-code-3.27.2\out\dist\debug.js    dart_test
[01:32:21] [DAP] [Info] Starting debug session 5f4600cc-bb0e-455f-9d14-c5a9cce772f9
[01:32:21] [DAP] [Info] <== {"command":"initialize","arguments":{"clientID":"vscode","clientName":"Visual Studio Code","adapterID":"dart","pathFormat":"path","linesStartAt1":true,"columnsStartAt1":true,"supportsVariableType":true,"supportsVariablePaging":true,"supportsRunInTerminalRequest":true,"locale":"en-us","supportsProgressReporting":true,"supportsInvalidatedEvent":true,"supportsMemoryReferences":true},"type":"request","seq":1}
[01:32:21] [DAP] [Info] ==> {"seq":1,"type":"response","request_seq":1,"command":"initialize","success":true,"body":{"supportsConfigurationDoneRequest":true,"supportsEvaluateForHovers":true,"supportsDelayedStackTraceLoading":true,"supportsConditionalBreakpoints":true,"supportsLogPoints":true,"supportsTerminateRequest":true,"supportsRestartFrame":true,"supportsClipboardContext":true,"exceptionBreakpointFilters":[{"filter":"All","label":"All Exceptions","default":false},{"filter":"Unhandled","label":"Uncaught Exceptions","default":true}]}}
[01:32:21] [DAP] [Info] <== {"command":"launch","arguments":{"suppressPromptOnErrors":false,"name":"Tests simple_test.dart","noDebug":true,"request":"launch","type":"dart","expectSingleTest":true,"program":"C:\\Temp\\test_test\\test\\simple_test.dart","toolArgs":["--name","^First test( \\(variant: .*\\))?$","--run-skipped"],"dartCodeDebugSessionID":"session-2e96","cwd":"C:\\Temp\\test_test","toolEnv":{"FLUTTER_HOST":"VSCode","PUB_ENVIRONMENT":"vscode.dart-code"},"sendLogsToClient":true,"additionalProjectPaths":["C:\\Temp\\test_test"],"args":[],"vmServicePort":0,"dartSdkPath":"C:\\flutter-2.5.3\\bin\\cache\\dart-sdk","maxLogLineLength":2000,"debugSdkLibraries":false,"debugExternalPackageLibraries":false,"showDartDeveloperLogs":true,"evaluateGettersInDebugViews":true,"evaluateToStringInDebugViews":true,"debuggerType":1,"__sessionId":"5f4600cc-bb0e-455f-9d14-c5a9cce772f9"},"type":"request","seq":2}
[01:32:21] [DAP] [Info] ==> {"seq":2,"type":"event","event":"dart.progressStart","body":{"progressID":"launch","message":"Launching…"}}
[01:32:21] [General] [Warn] Did not find session 5f4600cc-bb0e-455f-9d14-c5a9cce772f9 to handle dart.progressStart. There were 0 sessions:
[01:32:21] [General] [Warn] Event will be queued and processed when the session start event fires
[01:32:21] [DAP] [Info] ==> {"seq":3,"type":"response","request_seq":2,"command":"launch","success":true}
[01:32:21] [DAP] [Info] ==> {"seq":4,"type":"event","event":"dart.log","body":{"message":"Spawning C:\\flutter-2.5.3\\bin\\cache\\dart-sdk\\bin\\dart.exe with args [\"run\",\"--no-serve-devtools\",\"test:test\",\"-r\",\"json\",\"-j1\",\"--name\",\"^First test( \\\\(variant: .*\\\\))?$\",\"--run-skipped\",\"test\\\\simple_test.dart\"]","severity":0,"category":6}}
[01:32:21] [DAP] [Info] ==> {"seq":5,"type":"event","event":"dart.log","body":{"message":"..  in C:\\Temp\\test_test","severity":0,"category":6}}
[01:32:21] [DAP] [Info] ==> {"seq":6,"type":"event","event":"dart.log","body":{"message":"..  with {\"toolEnv\":{\"FLUTTER_HOST\":\"VSCode\",\"PUB_ENVIRONMENT\":\"vscode.dart-code\"}}","severity":0,"category":6}}
[01:32:21] [General] [Info] Processing delayed event dart.progressStart for session 5f4600cc-bb0e-455f-9d14-c5a9cce772f9
[01:32:21] [PubTest] [Info] [Tests simple_test.dart] Spawning C:\flutter-2.5.3\bin\cache\dart-sdk\bin\dart.exe with args ["run","--no-serve-devtools","test:test","-r","json","-j1","--name","^First test( \\(variant: .*\\))?$","--run-skipped","test\\simple_test.dart"]
[01:32:21] [PubTest] [Info] [Tests simple_test.dart] ..  in C:\Temp\test_test
[01:32:21] [PubTest] [Info] [Tests simple_test.dart] ..  with {"toolEnv":{"FLUTTER_HOST":"VSCode","PUB_ENVIRONMENT":"vscode.dart-code"}}
[01:32:21] [DAP] [Info] ==> {"seq":7,"type":"event","event":"dart.log","body":{"message":"    PID: 22284","severity":0,"category":6}}
[01:32:21] [DAP] [Info] ==> {"seq":8,"type":"event","event":"dart.progressEnd","body":{"progressID":"launch"}}
[01:32:21] [DAP] [Info] ==> {"seq":9,"type":"event","event":"initialized"}
[01:32:21] [DAP] [Info] <== {"command":"configurationDone","type":"request","seq":3}
[01:32:21] [PubTest] [Info] [Tests simple_test.dart]     PID: 22284
[01:32:21] [DAP] [Info] ==> {"seq":10,"type":"response","request_seq":3,"command":"configurationDone","success":true}
[01:32:21] [DAP] [Info] <== {"command":"threads","type":"request","seq":4}
[01:32:21] [DAP] [Info] ==> {"seq":11,"type":"response","request_seq":4,"command":"threads","success":true,"body":{"threads":[]}}
[01:32:23] [DAP] [Info] ==> {"seq":12,"type":"event","event":"dart.log","body":{"message":"<== {\"protocolVersion\":\"0.1.1\",\"runnerVersion\":\"1.19.2\",\"pid\":24432,\"type\":\"start\",\"time\":0}\r\n","severity":0,"category":6}}
[01:32:23] [PubTest] [Info] [Tests simple_test.dart] <== {"protocolVersion":"0.1.1","runnerVersion":"1.19.2","pid":24432,"type":"start","time":0}
[01:32:23] [DAP] [Info] ==> {"seq":13,"type":"event","event":"dart.log","body":{"message":"<== {\"suite\":{\"id\":0,\"platform\":\"vm\",\"path\":\"test\\\\simple_test.dart\"},\"type\":\"suite\",\"time\":0}\r\n","severity":0,"category":6}}
[01:32:23] [DAP] [Info] ==> {"seq":14,"type":"event","event":"dart.testNotification","body":{"suite":{"id":0,"platform":"vm","path":"C:\\Temp\\test_test\\test\\simple_test.dart"},"type":"suite","time":0}}
[01:32:23] [DAP] [Info] ==> {"seq":15,"type":"event","event":"dart.log","body":{"message":"<== {\"test\":{\"id\":1,\"name\":\"loading test\\\\simple_test.dart\",\"suiteID\":0,\"groupIDs\":[],\"metadata\":{\"skip\":false,\"skipReason\":null},\"line\":null,\"column\":null,\"url\":null},\"type\":\"testStart\",\"time\":4}\r\n","severity":0,"category":6}}
[01:32:23] [DAP] [Info] ==> {"seq":16,"type":"event","event":"dart.testNotification","body":{"test":{"id":1,"name":"loading test\\simple_test.dart","suiteID":0,"groupIDs":[],"metadata":{"skip":false,"skipReason":null},"line":null,"column":null,"url":null},"type":"testStart","time":4}}
[01:32:23] [PubTest] [Info] [Tests simple_test.dart] <== {"suite":{"id":0,"platform":"vm","path":"test\\simple_test.dart"},"type":"suite","time":0}
[01:32:23] [PubTest] [Info] [Tests simple_test.dart] <== {"test":{"id":1,"name":"loading test\\simple_test.dart","suiteID":0,"groupIDs":[],"metadata":{"skip":false,"skipReason":null},"line":null,"column":null,"url":null},"type":"testStart","time":4}
[01:32:23] [DAP] [Info] ==> {"seq":17,"type":"event","event":"dart.log","body":{"message":"<== {\"count\":1,\"time\":19,\"type\":\"allSuites\"}\r\n","severity":0,"category":6}}
[01:32:23] [PubTest] [Info] [Tests simple_test.dart] <== {"count":1,"time":19,"type":"allSuites"}
[01:32:24] [DAP] [Info] ==> {"seq":18,"type":"event","event":"dart.log","body":{"message":"<== {\"testID\":1,\"result\":\"success\",\"skipped\":false,\"hidden\":true,\"type\":\"testDone\",\"time\":1157}\r\n","severity":0,"category":6}}
[01:32:24] [DAP] [Info] ==> {"seq":19,"type":"event","event":"dart.testNotification","body":{"testID":1,"result":"success","skipped":false,"hidden":true,"type":"testDone","time":1157}}
[01:32:24] [PubTest] [Info] [Tests simple_test.dart] <== {"testID":1,"result":"success","skipped":false,"hidden":true,"type":"testDone","time":1157}
[01:32:24] [DAP] [Info] ==> {"seq":20,"type":"event","event":"dart.log","body":{"message":"<== {\"success\":true,\"type\":\"done\",\"time\":1184}\r\n","severity":0,"category":6}}
[01:32:24] [PubTest] [Info] [Tests simple_test.dart] <== {"success":true,"type":"done","time":1184}
[01:32:24] [DAP] [Info] ==> {"seq":21,"type":"event","event":"dart.log","body":{"message":"No tests match regular expression \"^First test( \\(variant: .*\\))?$\".","severity":2,"category":6}}
[01:32:24] [DAP] [Info] ==> {"seq":22,"type":"event","event":"dart.log","body":{"message":"\n","severity":2,"category":6}}
[01:32:24] [DAP] [Info] ==> {"seq":23,"type":"event","event":"output","body":{"category":"stderr","output":"No tests match regular expression \"^First test( \\(variant: .*\\))?$\".\n"}}
[01:32:24] [PubTest] [Error] [Tests simple_test.dart] No tests match regular expression "^First test( \(variant: .*\))?$".
[01:32:24] [PubTest] [Error] [Tests simple_test.dart]
[01:32:24] [DAP] [Info] ==> {"seq":24,"type":"event","event":"dart.log","body":{"message":"Process terminated! 79, null","severity":0,"category":6}}
[01:32:24] [DAP] [Info] ==> {"seq":25,"type":"event","event":"dart.log","body":{"message":"Process exited (79)","severity":0,"category":10}}
[01:32:24] [DAP] [Info] ==> {"seq":26,"type":"event","event":"output","body":{"category":"console","output":"Exited (79)\n"}}
[01:32:24] [PubTest] [Info] [Tests simple_test.dart] Process terminated! 79, null
[01:32:24] [VmService] [Info] [Tests simple_test.dart] Process exited (79)
[01:32:24] [DAP] [Info] ==> {"seq":27,"type":"event","event":"terminated"}
[01:32:24] [DAP] [Info] <== {"command":"disconnect","arguments":{"restart":false},"type":"request","seq":5}
[01:32:24] [DAP] [Info] ==> {"seq":28,"type":"event","event":"dart.log","body":{"message":"Disconnect requested!","severity":0,"category":10}}
[01:32:24] [DAP] [Info] ==> {"seq":29,"type":"event","event":"dart.log","body":{"message":"TERM: Requested to terminate with SIGINT...","severity":0,"category":10}}
[01:32:24] [DAP] [Info] ==> {"seq":30,"type":"event","event":"dart.log","body":{"message":"TERM: Did not need to terminate processes","severity":0,"category":10}}
[01:32:24] [DAP] [Info] ==> {"seq":31,"type":"event","event":"dart.log","body":{"message":"TERM: Removing all stored data...","severity":0,"category":10}}
[01:32:24] [DAP] [Info] ==> {"seq":32,"type":"event","event":"dart.log","body":{"message":"TERM: Waiting for process to finish...","severity":0,"category":10}}
[01:32:24] [DAP] [Info] ==> {"seq":33,"type":"event","event":"dart.log","body":{"message":"TERM: Disconnecting...","severity":0,"category":10}}
[01:32:24] [VmService] [Info] [Tests simple_test.dart] Disconnect requested!
[01:32:24] [VmService] [Info] [Tests simple_test.dart] TERM: Requested to terminate with SIGINT...
[01:32:24] [VmService] [Info] [Tests simple_test.dart] TERM: Did not need to terminate processes
[01:32:24] [VmService] [Info] [Tests simple_test.dart] TERM: Removing all stored data...
[01:32:24] [VmService] [Info] [Tests simple_test.dart] TERM: Waiting for process to finish...
[01:32:24] [VmService] [Info] [Tests simple_test.dart] TERM: Disconnecting...
[01:32:24] [DAP] [Info] ==> {"seq":34,"type":"response","request_seq":5,"command":"disconnect","success":true}
[01:32:24] [DAP] [Info] Debug session 5f4600cc-bb0e-455f-9d14-c5a9cce772f9 errored: {}
[01:32:24] [DAP] [Info] Debug session 5f4600cc-bb0e-455f-9d14-c5a9cce772f9 exit: code: 0, signal: undefined
[01:32:25] [DAP] [Info] Stopping debug session 5f4600cc-bb0e-455f-9d14-c5a9cce772f9
[01:32:25] [General] [Info] Debugger:SessionDuration timing: 3648ms (PubTest)
Tue Oct 26 2021 [01:32:25 GMT-0300 (Horário Padrão de Brasília)] Log file ended

!! PLEASE REVIEW THIS LOG FOR SENSITIVE INFORMATION BEFORE SHARING !!

Dart Code extension: 3.27.2
Flutter extension: 3.27.0 (not activated)

App: Visual Studio Code
Version: 1.61.2
Platform: win

Workspace type: Dart
Analyzer type: LSP
Multi-root?: false

Dart SDK:
    Loc: C:\flutter-2.5.3\bin\cache\dart-sdk
    Ver: 2.14.4
Flutter SDK:
    Loc: C:\flutter-2.5.3
    Ver: 2.5.3

HTTP_PROXY: undefined
NO_PROXY: undefined

Logging Categories:
    General, CommandProcesses, DAP, VmService, DevTools, FlutterDaemon, FlutterRun, FlutterTest, PubTest, WebDaemon

Tue Oct 26 2021 [01:32:07 GMT-0300 (Horário Padrão de Brasília)] Log file started
[01:32:12] [General] [Info] Starting debug session...
[01:32:12] [General] [Info]     workspace: C:\Temp\test_test
[01:32:12] [General] [Info]     program  : C:\Temp\test_test\test\simple_test.dart
[01:32:12] [General] [Info] Using workspace as cwd: C:\Temp\test_test
[01:32:12] [General] [Info] Project (C:\Temp\test_test\test\simple_test.dart) not recognised as Flutter or Web, will use Dart debugger
[01:32:12] [General] [Info] Detected launch project as Dart
[01:32:12] [General] [Info] Detected launch project as a Test project
[01:32:12] [General] [Info] Using PubTest debug adapter for this session
[01:32:12] [General] [Info] Checking for errors before launching
[01:32:12] [General] [Info] Debug session starting...
    {
        "suppressPromptOnErrors": false,
        "name": "Tests simple_test.dart",
        "noDebug": true,
        "request": "launch",
        "type": "dart",
        "expectSingleTest": true,
        "program": "C:\\Temp\\test_test\\test\\simple_test.dart",
        "toolArgs": [
            "--name",
            "^Second group Second test( \\(variant: .*\\))?$",
            "--run-skipped"
        ],
        "dartCodeDebugSessionID": "session-ced0",
        "cwd": "C:\\Temp\\test_test",
        "toolEnv": {
            "FLUTTER_HOST": "VSCode",
            "PUB_ENVIRONMENT": "vscode.dart-code"
        },
        "sendLogsToClient": true,
        "additionalProjectPaths": [
            "C:\\Temp\\test_test"
        ],
        "args": [],
        "vmServicePort": 0,
        "dartSdkPath": "C:\\flutter-2.5.3\\bin\\cache\\dart-sdk",
        "maxLogLineLength": 2000,
        "debugSdkLibraries": false,
        "debugExternalPackageLibraries": false,
        "showDartDeveloperLogs": true,
        "evaluateGettersInDebugViews": true,
        "evaluateToStringInDebugViews": true,
        "debuggerType": 1
    }
[01:32:12] [General] [Info] Using dart_test debugger for PubTest
[01:32:12] [General] [Info] Running debugger via node with c:\.vscode\extensions\dart-code.dart-code-3.27.2\out\dist\debug.js    dart_test
[01:32:12] [DAP] [Info] Starting debug session 761bae32-ee27-4954-8ce6-ddec266b8b30
[01:32:12] [DAP] [Info] <== {"command":"initialize","arguments":{"clientID":"vscode","clientName":"Visual Studio Code","adapterID":"dart","pathFormat":"path","linesStartAt1":true,"columnsStartAt1":true,"supportsVariableType":true,"supportsVariablePaging":true,"supportsRunInTerminalRequest":true,"locale":"en-us","supportsProgressReporting":true,"supportsInvalidatedEvent":true,"supportsMemoryReferences":true},"type":"request","seq":1}
[01:32:12] [DAP] [Info] ==> {"seq":1,"type":"response","request_seq":1,"command":"initialize","success":true,"body":{"supportsConfigurationDoneRequest":true,"supportsEvaluateForHovers":true,"supportsDelayedStackTraceLoading":true,"supportsConditionalBreakpoints":true,"supportsLogPoints":true,"supportsTerminateRequest":true,"supportsRestartFrame":true,"supportsClipboardContext":true,"exceptionBreakpointFilters":[{"filter":"All","label":"All Exceptions","default":false},{"filter":"Unhandled","label":"Uncaught Exceptions","default":true}]}}
[01:32:12] [DAP] [Info] <== {"command":"launch","arguments":{"suppressPromptOnErrors":false,"name":"Tests simple_test.dart","noDebug":true,"request":"launch","type":"dart","expectSingleTest":true,"program":"C:\\Temp\\test_test\\test\\simple_test.dart","toolArgs":["--name","^Second group Second test( \\(variant: .*\\))?$","--run-skipped"],"dartCodeDebugSessionID":"session-ced0","cwd":"C:\\Temp\\test_test","toolEnv":{"FLUTTER_HOST":"VSCode","PUB_ENVIRONMENT":"vscode.dart-code"},"sendLogsToClient":true,"additionalProjectPaths":["C:\\Temp\\test_test"],"args":[],"vmServicePort":0,"dartSdkPath":"C:\\flutter-2.5.3\\bin\\cache\\dart-sdk","maxLogLineLength":2000,"debugSdkLibraries":false,"debugExternalPackageLibraries":false,"showDartDeveloperLogs":true,"evaluateGettersInDebugViews":true,"evaluateToStringInDebugViews":true,"debuggerType":1,"__sessionId":"761bae32-ee27-4954-8ce6-ddec266b8b30"},"type":"request","seq":2}
[01:32:12] [DAP] [Info] ==> {"seq":2,"type":"event","event":"dart.progressStart","body":{"progressID":"launch","message":"Launching…"}}
[01:32:12] [General] [Warn] Did not find session 761bae32-ee27-4954-8ce6-ddec266b8b30 to handle dart.progressStart. There were 0 sessions:
[01:32:12] [General] [Warn] Event will be queued and processed when the session start event fires
[01:32:12] [DAP] [Info] ==> {"seq":3,"type":"response","request_seq":2,"command":"launch","success":true}
[01:32:12] [DAP] [Info] ==> {"seq":4,"type":"event","event":"dart.log","body":{"message":"Spawning C:\\flutter-2.5.3\\bin\\cache\\dart-sdk\\bin\\dart.exe with args [\"run\",\"--no-serve-devtools\",\"test:test\",\"-r\",\"json\",\"-j1\",\"--name\",\"^Second group Second test( \\\\(variant: .*\\\\))?$\",\"--run-skipped\",\"test\\\\simple_test.dart\"]","severity":0,"category":6}}
[01:32:12] [DAP] [Info] ==> {"seq":5,"type":"event","event":"dart.log","body":{"message":"..  in C:\\Temp\\test_test","severity":0,"category":6}}
[01:32:12] [DAP] [Info] ==> {"seq":6,"type":"event","event":"dart.log","body":{"message":"..  with {\"toolEnv\":{\"FLUTTER_HOST\":\"VSCode\",\"PUB_ENVIRONMENT\":\"vscode.dart-code\"}}","severity":0,"category":6}}
[01:32:12] [General] [Info] Processing delayed event dart.progressStart for session 761bae32-ee27-4954-8ce6-ddec266b8b30
[01:32:12] [PubTest] [Info] [Tests simple_test.dart] Spawning C:\flutter-2.5.3\bin\cache\dart-sdk\bin\dart.exe with args ["run","--no-serve-devtools","test:test","-r","json","-j1","--name","^Second group Second test( \\(variant: .*\\))?$","--run-skipped","test\\simple_test.dart"]
[01:32:12] [PubTest] [Info] [Tests simple_test.dart] ..  in C:\Temp\test_test
[01:32:12] [PubTest] [Info] [Tests simple_test.dart] ..  with {"toolEnv":{"FLUTTER_HOST":"VSCode","PUB_ENVIRONMENT":"vscode.dart-code"}}
[01:32:12] [DAP] [Info] ==> {"seq":7,"type":"event","event":"dart.log","body":{"message":"    PID: 28156","severity":0,"category":6}}
[01:32:12] [DAP] [Info] ==> {"seq":8,"type":"event","event":"dart.progressEnd","body":{"progressID":"launch"}}
[01:32:12] [DAP] [Info] ==> {"seq":9,"type":"event","event":"initialized"}
[01:32:12] [DAP] [Info] <== {"command":"configurationDone","type":"request","seq":3}
[01:32:12] [PubTest] [Info] [Tests simple_test.dart]     PID: 28156
[01:32:12] [DAP] [Info] ==> {"seq":10,"type":"response","request_seq":3,"command":"configurationDone","success":true}
[01:32:12] [DAP] [Info] <== {"command":"threads","type":"request","seq":4}
[01:32:12] [DAP] [Info] ==> {"seq":11,"type":"response","request_seq":4,"command":"threads","success":true,"body":{"threads":[]}}
[01:32:13] [DAP] [Info] ==> {"seq":12,"type":"event","event":"dart.log","body":{"message":"<== {\"protocolVersion\":\"0.1.1\",\"runnerVersion\":\"1.19.2\",\"pid\":23296,\"type\":\"start\",\"time\":0}\r\n","severity":0,"category":6}}
[01:32:13] [PubTest] [Info] [Tests simple_test.dart] <== {"protocolVersion":"0.1.1","runnerVersion":"1.19.2","pid":23296,"type":"start","time":0}
[01:32:14] [DAP] [Info] ==> {"seq":13,"type":"event","event":"dart.log","body":{"message":"<== {\"suite\":{\"id\":0,\"platform\":\"vm\",\"path\":\"test\\\\simple_test.dart\"},\"type\":\"suite\",\"time\":1}\r\n","severity":0,"category":6}}
[01:32:14] [DAP] [Info] ==> {"seq":14,"type":"event","event":"dart.testNotification","body":{"suite":{"id":0,"platform":"vm","path":"C:\\Temp\\test_test\\test\\simple_test.dart"},"type":"suite","time":1}}
[01:32:14] [PubTest] [Info] [Tests simple_test.dart] <== {"suite":{"id":0,"platform":"vm","path":"test\\simple_test.dart"},"type":"suite","time":1}
[01:32:14] [DAP] [Info] ==> {"seq":15,"type":"event","event":"dart.log","body":{"message":"<== {\"test\":{\"id\":1,\"name\":\"loading test\\\\simple_test.dart\",\"suiteID\":0,\"groupIDs\":[],\"metadata\":{\"skip\":false,\"skipReason\":null},\"line\":null,\"column\":null,\"url\":null},\"type\":\"testStart\",\"time\":8}\r\n","severity":0,"category":6}}
[01:32:14] [DAP] [Info] ==> {"seq":16,"type":"event","event":"dart.testNotification","body":{"test":{"id":1,"name":"loading test\\simple_test.dart","suiteID":0,"groupIDs":[],"metadata":{"skip":false,"skipReason":null},"line":null,"column":null,"url":null},"type":"testStart","time":8}}
[01:32:14] [PubTest] [Info] [Tests simple_test.dart] <== {"test":{"id":1,"name":"loading test\\simple_test.dart","suiteID":0,"groupIDs":[],"metadata":{"skip":false,"skipReason":null},"line":null,"column":null,"url":null},"type":"testStart","time":8}
[01:32:14] [DAP] [Info] ==> {"seq":17,"type":"event","event":"dart.log","body":{"message":"<== {\"count\":1,\"time\":25,\"type\":\"allSuites\"}\r\n","severity":0,"category":6}}
[01:32:14] [PubTest] [Info] [Tests simple_test.dart] <== {"count":1,"time":25,"type":"allSuites"}
[01:32:15] [DAP] [Info] ==> {"seq":18,"type":"event","event":"dart.log","body":{"message":"<== {\"testID\":1,\"result\":\"success\",\"skipped\":false,\"hidden\":true,\"type\":\"testDone\",\"time\":1045}\r\n","severity":0,"category":6}}
[01:32:15] [DAP] [Info] ==> {"seq":19,"type":"event","event":"dart.testNotification","body":{"testID":1,"result":"success","skipped":false,"hidden":true,"type":"testDone","time":1045}}
[01:32:15] [PubTest] [Info] [Tests simple_test.dart] <== {"testID":1,"result":"success","skipped":false,"hidden":true,"type":"testDone","time":1045}
[01:32:15] [DAP] [Info] ==> {"seq":20,"type":"event","event":"dart.log","body":{"message":"<== {\"group\":{\"id\":2,\"suiteID\":0,\"parentID\":null,\"name\":\"\",\"metadata\":{\"skip\":false,\"skipReason\":null},\"testCount\":1,\"line\":null,\"column\":null,\"url\":null},\"type\":\"group\",\"time\":1055}\r\n","severity":0,"category":6}}
[01:32:15] [DAP] [Info] ==> {"seq":21,"type":"event","event":"dart.testNotification","body":{"group":{"id":2,"suiteID":0,"parentID":null,"name":"","metadata":{"skip":false,"skipReason":null},"testCount":1,"line":null,"column":null,"url":null},"type":"group","time":1055}}
[01:32:15] [DAP] [Info] ==> {"seq":22,"type":"event","event":"dart.log","body":{"message":"<== {\"group\":{\"id\":3,\"suiteID\":0,\"parentID\":2,\"name\":\"Second group\",\"metadata\":{\"skip\":false,\"skipReason\":null},\"testCount\":1,\"line\":5,\"column\":3,\"url\":\"file:///C:/Temp/test_test/test/simple_test.dart\"},\"type\":\"group\",\"time\":1057}\r\n","severity":0,"category":6}}
[01:32:15] [DAP] [Info] ==> {"seq":23,"type":"event","event":"dart.testNotification","body":{"group":{"id":3,"suiteID":0,"parentID":2,"name":"Second group","metadata":{"skip":false,"skipReason":null},"testCount":1,"line":5,"column":3,"url":"file:///C:/Temp/test_test/test/simple_test.dart"},"type":"group","time":1057}}
[01:32:15] [DAP] [Info] ==> {"seq":24,"type":"event","event":"dart.log","body":{"message":"<== {\"test\":{\"id\":4,\"name\":\"Second group Second test\",\"suiteID\":0,\"groupIDs\":[2,3],\"metadata\":{\"skip\":false,\"skipReason\":null},\"line\":6,\"column\":5,\"url\":\"file:///C:/Temp/test_test/test/simple_test.dart\"},\"type\":\"testStart\",\"time\":1057}\r\n","severity":0,"category":6}}
[01:32:15] [DAP] [Info] ==> {"seq":25,"type":"event","event":"dart.testNotification","body":{"test":{"id":4,"name":"Second group Second test","suiteID":0,"groupIDs":[2,3],"metadata":{"skip":false,"skipReason":null},"line":6,"column":5,"url":"file:///C:/Temp/test_test/test/simple_test.dart"},"type":"testStart","time":1057}}
[01:32:15] [PubTest] [Info] [Tests simple_test.dart] <== {"group":{"id":2,"suiteID":0,"parentID":null,"name":"","metadata":{"skip":false,"skipReason":null},"testCount":1,"line":null,"column":null,"url":null},"type":"group","time":1055}
[01:32:15] [PubTest] [Info] [Tests simple_test.dart] <== {"group":{"id":3,"suiteID":0,"parentID":2,"name":"Second group","metadata":{"skip":false,"skipReason":null},"testCount":1,"line":5,"column":3,"url":"file:///C:/Temp/test_test/test/simple_test.dart"},"type":"group","time":1057}
[01:32:15] [PubTest] [Info] [Tests simple_test.dart] <== {"test":{"id":4,"name":"Second group Second test","suiteID":0,"groupIDs":[2,3],"metadata":{"skip":false,"skipReason":null},"line":6,"column":5,"url":"file:///C:/Temp/test_test/test/simple_test.dart"},"type":"testStart","time":1057}
[01:32:15] [DAP] [Info] ==> {"seq":26,"type":"event","event":"dart.log","body":{"message":"<== {\"testID\":4,\"result\":\"success\",\"skipped\":false,\"hidden\":false,\"type\":\"testDone\",\"time\":1100}\r\n","severity":0,"category":6}}
[01:32:15] [DAP] [Info] ==> {"seq":27,"type":"event","event":"output","body":{"category":"stdout","output":"✓ Second group Second test\n"}}
[01:32:15] [DAP] [Info] ==> {"seq":28,"type":"event","event":"dart.testNotification","body":{"testID":4,"result":"success","skipped":false,"hidden":false,"type":"testDone","time":1100}}
[01:32:15] [PubTest] [Info] [Tests simple_test.dart] <== {"testID":4,"result":"success","skipped":false,"hidden":false,"type":"testDone","time":1100}
[01:32:15] [DAP] [Info] ==> {"seq":29,"type":"event","event":"dart.log","body":{"message":"<== {\"success\":true,\"type\":\"done\",\"time\":1124}\r\n","severity":0,"category":6}}
[01:32:15] [PubTest] [Info] [Tests simple_test.dart] <== {"success":true,"type":"done","time":1124}
[01:32:15] [DAP] [Info] ==> {"seq":30,"type":"event","event":"dart.log","body":{"message":"Process terminated! 0, null","severity":0,"category":6}}
[01:32:15] [DAP] [Info] ==> {"seq":31,"type":"event","event":"dart.log","body":{"message":"Process exited (0)","severity":0,"category":10}}
[01:32:15] [DAP] [Info] ==> {"seq":32,"type":"event","event":"output","body":{"category":"console","output":"Exited\n"}}
[01:32:15] [PubTest] [Info] [Tests simple_test.dart] Process terminated! 0, null
[01:32:15] [VmService] [Info] [Tests simple_test.dart] Process exited (0)
[01:32:15] [DAP] [Info] ==> {"seq":33,"type":"event","event":"terminated"}
[01:32:15] [DAP] [Info] <== {"command":"disconnect","arguments":{"restart":false},"type":"request","seq":5}
[01:32:15] [DAP] [Info] ==> {"seq":34,"type":"event","event":"dart.log","body":{"message":"Disconnect requested!","severity":0,"category":10}}
[01:32:15] [DAP] [Info] ==> {"seq":35,"type":"event","event":"dart.log","body":{"message":"TERM: Requested to terminate with SIGINT...","severity":0,"category":10}}
[01:32:15] [DAP] [Info] ==> {"seq":36,"type":"event","event":"dart.log","body":{"message":"TERM: Did not need to terminate processes","severity":0,"category":10}}
[01:32:15] [DAP] [Info] ==> {"seq":37,"type":"event","event":"dart.log","body":{"message":"TERM: Removing all stored data...","severity":0,"category":10}}
[01:32:15] [DAP] [Info] ==> {"seq":38,"type":"event","event":"dart.log","body":{"message":"TERM: Waiting for process to finish...","severity":0,"category":10}}
[01:32:15] [DAP] [Info] ==> {"seq":39,"type":"event","event":"dart.log","body":{"message":"TERM: Disconnecting...","severity":0,"category":10}}
[01:32:15] [VmService] [Info] [Tests simple_test.dart] Disconnect requested!
[01:32:15] [VmService] [Info] [Tests simple_test.dart] TERM: Requested to terminate with SIGINT...
[01:32:15] [VmService] [Info] [Tests simple_test.dart] TERM: Did not need to terminate processes
[01:32:15] [VmService] [Info] [Tests simple_test.dart] TERM: Removing all stored data...
[01:32:15] [VmService] [Info] [Tests simple_test.dart] TERM: Waiting for process to finish...
[01:32:15] [VmService] [Info] [Tests simple_test.dart] TERM: Disconnecting...
[01:32:15] [DAP] [Info] ==> {"seq":40,"type":"response","request_seq":5,"command":"disconnect","success":true}
[01:32:15] [DAP] [Info] Debug session 761bae32-ee27-4954-8ce6-ddec266b8b30 errored: {}
[01:32:15] [DAP] [Info] Debug session 761bae32-ee27-4954-8ce6-ddec266b8b30 exit: code: 0, signal: undefined
[01:32:15] [DAP] [Info] Stopping debug session 761bae32-ee27-4954-8ce6-ddec266b8b30
[01:32:16] [General] [Info] Debugger:SessionDuration timing: 3507ms (PubTest)
Tue Oct 26 2021 [01:32:16 GMT-0300 (Horário Padrão de Brasília)] Log file ended

The text was updated successfully, but these errors were encountered:

DanTup · 2021-10-26T10:05:54Z

Yes, this is an unfortunate limitation. The issue is that VS Code runs tests by providing the "full test name" (which is all groups+test name combined) to dart test. In your non-working example above, the structure of the file makes it seem like "First test" is the complete test name, but it's not so it does not match.

If we changed to not use the full test name, then there are issues with code like this:

group('foo', () {
  test('does the expected', () {});
});
group('bar', () {
  test('does the expected', () {});
});

If we only passed "does the expected" then both tests would run when you tried to run either.

However, there may be a solution in the near future. package:test has recently gained the ability to filter tests by line number too, so we could in theory pass a partial test name and the line number and I think that might resolve this. We'll need to do something to ensure the version of package:test is new enough to support this, but I think that's doable.

…3630

DanTup · 2023-04-19T18:06:20Z

In the next release (and the pre-release version just pushed today - v3.63.20240419), you can choose to have tests executed using their line numbers rather than their names. This fixes a number of issues including this one, caused by not always being able to statically compute the exact name of a test.

Please let me know if you find any issues with this functionality!

jvretamero added the is enhancement label Oct 26, 2021

jvretamero changed the title ~~Tests with referenced function can't be run~~ Run tests with referenced function Oct 26, 2021

DanTup added this to the v3.29.0 milestone Oct 26, 2021

DanTup added the in testing Relates to test execution of Dart/Flutter tests for end users label Oct 26, 2021

DanTup changed the title ~~Run tests with referenced function~~ Support running single tests that are not inside their group hierarchy in the source file Oct 26, 2021

DanTup modified the milestones: v3.29.0, On Deck Nov 24, 2021

jvretamero added a commit to jvretamero/dart-excercism that referenced this issue Oct 10, 2022

Forth - Organizing tests to overcome this issue: Dart-Code/Dart-Code#…

18c7a23

…3630

DanTup modified the milestones: On Deck, v3.62.0 Feb 27, 2023

DanTup modified the milestones: v3.62.0, v3.64.0 Mar 30, 2023

DanTup closed this as completed Apr 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support running single tests that are not inside their group hierarchy in the source file #3630

Support running single tests that are not inside their group hierarchy in the source file #3630

jvretamero commented Oct 26, 2021

DanTup commented Oct 26, 2021

DanTup commented Apr 19, 2023

Support running single tests that are not inside their group hierarchy in the source file #3630

Support running single tests that are not inside their group hierarchy in the source file #3630

Comments

jvretamero commented Oct 26, 2021

DanTup commented Oct 26, 2021

DanTup commented Apr 19, 2023