Run tests using line/col to support groups/test with dynamic names

[MiniSuperDev]

Added by @DanTup:

Some test cases to ensure are handled:

• Run tests using line/col to support groups/test with dynamic names #4021
• Multiple tests execution #4150
• Parens around test name prevents CodeLens links from working #4168
• "No tests match regular expression...", customize regex #4099
• Debugging flutter tests fail if the descrription string has an interpolated value and a is composed of two concatenated strings #4250

---

Describe the bug
I have some functions to give a specific format to the descriptions of the groups and tests, like indentation, in this way it is easier to read them in the console, but when I execute a specific test, giving run in that group or test it tells me:

No tests match regular expression "^groupDescriptionFormatted\('group 1'\)".

To Reproduce
this is a minimal example:

import 'package:test/test.dart';

String groupDescriptionFormatted(String description) {
  return description;
}

String subGroupDescriptionFormatted(String description) {
  return '\n    $description';
}

void main() {
  group(
    groupDescriptionFormatted('group 1'),
    () {
      group(subGroupDescriptionFormatted('sub group 1'), () {
        test('test', () {
          expect(true, isTrue);
        });
      });
    },
  );
  group(
    groupDescriptionFormatted('group 2'),
    () {
      group(subGroupDescriptionFormatted('sub group 1'), () {
        test('test', () {
          expect(true, isTrue);
        });
      });
    },
  );
}

If you press run or debug on a specific group or test, you can see the error, an also in the testing tab you can see that the names are the function instead of the real test name.

If you press run or debug on main, or use the command run all tests, it runs normal but also in the testing tab keep the names are the function instead of the real test name, and now also the real tests names.

Thank you.

Please complete the following information:
Operating System and version: Windows 10
VS Code version: 1.68.0
Dart extension version: 3.42.0
Dart/Flutter SDK version: flutter 3.0.1

[DanTup]

Currently there are some limitations because of how tests are run individually (by name). Without executing the tests, it's impossible for the extension to know what groupDescriptionFormatted('group 1') returns, so it's not possible to build a dart test` command that has the correct test name to run.

However, once you've run the test file, you should be able to run the tests from the explorer, or from the test icons in the gutter, because the previous runs names would have been captured.

package:test does now have some other ways we can run tests (such as by line number/column) which may allow us to handle this better, but it'll need some work to move over (and ensure we don't try to do it for older versions of package:test).

I'll keep this issue open to track that, as it looks like there isn't an existing issue tracking it right now.

[jtdLab]

Will running single tests/groups with dynamic descriptions via vs code be possible with this?

[DanTup]

@jtdLab I'm hoping so, but I haven't done much testing yet (and it may depend on the exact configuration of the test). If you have some specific tests in mind, you can try running them from the terminal using the line/col like:

dart test "path/to/test.dart?line=10&col=2"

Where line/col are where your test definitions are. If that runs them, then it should work here. If it doesn't work, please post an example and I can do some digging.

[jtdLab]

@DanTup im thinking about a scenario like this

foo_test.dart

// I want to run this test via the vs code Run | Debug (even though it has no description)
rendersText(MyWidget(), text: 'Foo');

my_shared_testing_library.dart

@isTest
void rendersText(Widget widget, {required String text}) {
  testWidgets('renders $text', (tester) async {
    // Some testing here
  });
}

Is this possible via line/col and not description?