Don't use flags of the selected test preset when running an individual test

[quyykk]

Brief Issue Summary

I have an active selected test preset that filters out a couple of tests. Now I want to run a single test through the test explorer, that is not part of the test preset. But the extension executes ctest by passing the filter from the selected test preset, and so the result is that ctest can't find the test.

Running an individual test through the test explorer should not use any options/flags of the currently selected test preset.

CMake Tools Diagnostics

No response

Debug Log

No response

Additional Information

No response

[xisui-MSFT]

Sorry I'm not sure if I understand your question. If you want to use the test preset to filter out tests, why do you want to run tests that are filtered out? In addition, this is not supported by CTest either.

If I'm understanding this issue correctly, I'd suggest you have two presets. The first one contains all the configs and doesn't have any filters, and the second one inherits the first one and has filters. Then you'll able to use the first preset when trying to run any tests that are listed.

[quyykk]

I don't want to constantly switch presets just because I want to run one test.

But also this

Running an individual test through the test explorer should not use any options/flags of the currently selected test preset.

If I want to run a single test through the test explorer, then why are the options from the current preset used? The options from the current preset might be completely inappropriate for the single test run.

[xisui-MSFT]

I don't want to constantly switch presets just because I want to run one test.

If you are inheriting the test preset, then you can just use the base preset, and don't need to constantly switching them. Unless you have different settings for each test, in which case, unfortunately, I don't think there's a good way not switching presets.

The options from the current preset might be completely inappropriate for the single test run

As you said, this is why you shouldn't allow a preset include those tests that don't use the settings in this preset. Because otherwise we can't know what settings we should use for each test.

[quyykk]

What do you think about not specifying any options when running a single test through the test explorer?

[xisui-MSFT]

Then don't specify any options in your test preset? You can also use the default preset, which doesn't add anything.

But we don't do this by default. as you already implied in your earlier comments, some tests require additional settings. In another word, we chose to always list and run test under the selected test preset by default, since this is more generic. I think this is also what you asked in #3151.

[gcampbell-msft]

This issue is actually somewhat complex, and the root cause is that our design should only be displaying tests defined by the test preset in the Text Explorer. Currently, this is not working correctly and thus you're seeing tests in the Text Explorer that you shouldn't.

Therefore, this is likely more of a feature request, to be able to see all tests in the Text Explorer, even those that aren't defined by a test preset. Marking this as a feature request and we moved it to the 1.16 milestone.