Exclude Runtime Tests based on Runtime Mode [Interpreter/JIT/AOT]

[naricc]

We currently have the capability to exclude runtime tests using the issues.targets file, based on operating system, architecture, and runtime flavor (coreclr/mono).

There are some tests that fail on code generation or other mode of one of the runtimes, for example, mono-intepreter. Currently, to exclude them, we need to exclude the test for all runs on the runtime flavor.

This is a proposal to extend the issues.targets mechanism to handle these cases, to exclude tests only on particular configurations of the RuntimeFlavor.

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

We currently have the capability to exclude runtime tests using the issues.targets file, based on operating system, architecture, and runtime flavor (coreclr/mono).

There are some tests that fail on code generation or other mode of one of the runtimes, for example, mono-intepreter. Currently, to exclude them, we need to exclude the test for all runs on the runtime flavor.

This is a proposal to extend the issues.targets mechanism to handle these cases, to exclude tests only on particular configurations of the RuntimeFlavor.

Author:	naricc
Assignees:	naricc
Labels:	area-Infrastructure, test enhancement
Milestone:	-

[ViktorHofer]

cc @trylek

[naricc]

So I realized that, in order for this to work in CI, I will have to move the interpreter to a separate lane (as opposed to a scenario as it currently is). That is because issues.targets filters during the "Generate wrappers" phase, and therefore we need a separate helix payload for interpreter runs.

Is that likely to cause problems in terms of build resources or anything? @trylek @ViktorHofer @safern

[trylek]

I wouldn't expect too much negative fallout from splitting a Helix leg into two legs apart from the extra time for provisioning a VM and potentially higher risk of failure due to infra hiccups.

[naricc]

Implemented in this PR: #51522