Folders excluded via analysis_options.yaml should be treated the same as dart.analysisExcludedFolders, including exclusion from the test runner

[devoncarew]

Describe the bug

The Testing view in vs code performs auto test discovery. For one of our projects (https://github.com/dart-lang/dart-pad/tree/main/pkgs/dart_services, the backend of dartpad), it traverses down into several (very large) subdirectories where we wouldn't want tests discovered or run from.

One of the local directories is flutter-sdks/; this contains a copy of the flutter sdk.

Another is local_pub_cache/; this contains a sub-set of a local pub cache.

Both directories are ignored from the POV of git (via a .gitignore) and analysis (via an analysis options file - https://github.com/dart-lang/dart-pad/blob/main/pkgs/dart_services/analysis_options.yaml#L8).

I suspect the issue here is that we can't just look in test in order to handle cases where the user has opened several packages at once). Perhaps we could stop traversing down into dirs once we see a directory w/ a pubspec? Or, we could have the traverseral algorithm respect the glob excludes from the analysis options files.

Expected behavior

The Testing view should just show tests defined in the test directory, and have a way to exclude discovery in for portions of the project.

Please complete the following information:

• Dart 3.2.0-39.0.dev (dev) (Fri Aug 4 09:04:57 2023 -0700) on "macos_arm64"
• on macos / Version 13.4.1 (c) (Build 22F770820d)

[DanTup]

@devoncarew do you need to analyze those folders? If not, I think the dart.analysisExcludedFolders setting will exclude projects from test discovery (as well as analysis).

Perhaps we should consider exclusions in analysis_options too - although the reason we have our own setting is because it allows excluding large parts of trees that might not have analysis_options (node_modules), or where they might be deeply nested (the setting allows excluding further up).

The Testing view should just show tests defined in the test directory

Some tests go into integration_tests and some users put tests alongside the implementations (although we have a setting to control this). I'm not sure changing this helps here though, as presumably the tests you're seeing (but don't want) are in valid test folders for their respective projects (so it just looks like a nested project - something we should support)?

[devoncarew]

It sounds like the dart.analysisExcludedFolders setting would work here. However, I now have to exclude the directories (flutter-sdks/, local_pub_cache/) in three different places, and two overlap semantically (analysis_options excludes and analysisExcludedFolders).

[DanTup]

Yeah, I think it'd be nice if analysis_options excludes were merged into the exclude setting. For a long time I tried to avoid parsing these files that weren't owned by the extension (and avoiding a YAML parser since there didn't seem to be a first-party of de-facto one), but both those ships have sailed.

I'll rename this issue for handling exclusions from analysis_options.yaml.

[DanTup]

This change was included in a pre-release version (v3.71.20230829 which might take 20min or so to show up) if you want to try it out in advance of the release (probably Monday). Thanks!

[devoncarew]

Thanks! I'll check this out.