Improve discoverability of additional args for testing

[gaaclarke]

Is your feature request related to a problem? Please describe.

Yes, I'd like to execute the dart framework tests with a locally built engine.

Describe the solution you'd like

I'd like an entry in the Testing settings for DartCode that allows me to specify the arguments to make flutter use a local engine.

Describe alternatives you've considered

This would be the easiest solution.

Additional context

Here is how I invoke the test the way I want from the command line:

flutter --local-engine=host_debug_unopt_arm64 \
  --local-engine-host=host_debug_unopt_arm64 \
  --local-engine-src-path=/Users/aaclarke/dev/engine/src/ test \
  --name="Cupertino.*focusable\sand" \
  test/material/switch_test.dart 

[DanTup]

Does the dart.flutterTestAdditionalArgs do what you want?

[gaaclarke]

Does the dart.flutterTestAdditionalArgs do what you want?

Yes, that does! Thanks, I wish I had asked earlier. This is going to be really helpful. I was worried they would get added at the wrong part of the invocation but works like a charm.

I did look for something like this. I'm not sure if there is anything we could do to make it more findable. I was thrown off a bit about looking under the "Dart" settings but knowing I wanted "Flutter" settings.

We can't add something to the testing UI directly for it can we?

[DanTup]

Yeah, right now everything is under the Dart extension which is a little odd. There's a long-standing issue about moving all user-facing Flutter functionality to the Flutter extension so that it's clearer and if you do only pure Dart, you don't see Flutter anywhere. However, it may also add confusing if some settings are in Dart and Flutter users are expecting to find them in Flutter because they don't know where the distinction lies.

Although it's dart.x, the settings are in the Flutter section when viewing the UI:

We can't add something to the testing UI directly for it can we?

I'm not sure what you mean by "testing UI".. VS Code doesn't have a lot of UI. There are "profiles" for test runs, but they're things we have to define in the extension, so not something that would allow us to provide some UI to edit parameters from the testing UI to add your own flags.

You can also include them in toolArgs in a launch.json config instead of settings if that works better (the setting is just copied into the launch configuration when a session is being launched anyway).

[gaaclarke]

I'm not sure what you mean by "testing UI"

This is where I was talking about and where I would expect to find the option to modify the arguments for running tests. Probably under some ellipsis menu since it isn't the most common thing. I don't know if we have the ability to add anything there or if vscode just supplies the UI without modification.

[DanTup]

Ah, gotcha. I think we can contribute some commands there, although it's not clear to me what they would do. We could perhaps open the settings UI that I posted a screenshot of above, although we actually have have a dart.testAdditionalArgs that could be applicable if your workspace contains Dart + Flutter.

Generally, these kinds of settings may be better in a launch.json or in a Code-Workspace that you can save outside of the project, since if you commit .vscode/settings.json it might affect others using the the project.

[gaaclarke]

Opening the settings page and highlighting that variable would be an improvement. Editing the settings directly after interacting with a modal UI would be too. That's what happens when we use things like Tasks: Configure Default Build Task, but that's editing launch.json...

I know UX design and having contextual options is a better design, but you know vscode better than me so I defer to you about what we can do with vscode =)

[DanTup]

Let's re-open this and I'll try adding some items to that overflow for menu for something like "Edit 'dart test' args" and "Edit 'flutter test' args" that jump to those settings and see how it feels.