Enable use of RuntimeVariant to exclude runtime tests

[naricc]

Currently, we can only exclude tests based on architecture, runtime flavor (mono/coreclr), and operating system. This change additionally allows the exclusions of tests based on runtime variants, such as interpreter, jit, or llvmaot.

I haven't gotten a full green build out of this yet, but I believe it trying to run the right things now. So I am making it ready for review.

Fixes #49653

[ghost]

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Currently, we can only exclude tests based on architecture, runtime flavor (mono/coreclr), and operating system. This change additionally allows the exclusions of tests based on runtime variants, such as interpreter, jit, or llvmaot.

This is in draft while I work out the CI parts of it.

Author:	naricc
Assignees:	-
Labels:	area-Infrastructure-mono
Milestone:	-

[SamMonoRT]

@naricc - while thinking about this, also consider the possibility of adding an extra refinement for differentiating Interpreter WASM v/s AOT WASM test runs. Quite likely the set of failures maybe different.

[lambdageek]

🎉

[naricc]

@SamMonoRT Yeah, I had considered that. For interpreter wasm we can already just look at the architecture and variant; same for aot. The mechanism is general enough that we should be able to use it for any of the jit/aot/aotllvm things we have, just by defining a new variant for each.

It would be nice if I could create an enum for this, but there is no such facility in msbuild, or any way to statically check that the names agree with what it is in the build yml files.

[radical]

Can we surface this in PlatformDetection (or similar) so that it can be used with attributes like ActiveIssue? You will probably have to use #ifdef FOO+msbuild to define FOO. It would be useful here.

[radical]

I was trying https://gist.github.com/radical/dcbadb80909bd78fbabc4020a6018298#file-foo-diff-L74-L96, but that wouldn't be enough.

[naricc]

@radical Maybe. I am not sure what you had in mind exactly, adding a "GetRuntimeVariant" method to PlatformDetection?

Currently this is only for the runtime tests, which have their own unique way of building and don't use PlatfromDetection.

[radical]

@radical Maybe. I am not sure what you had in mind exactly, adding a "GetRuntimeVariant" method to PlatformDetection?

Currently this is only for the runtime tests, which have their own unique way of building and don't use PlatfromDetection.

Oh ok, I was mistaken then. I'll add something for the library tests.

[trylek]

LGTM, thank you!

[imhameed]

_il_relu_conv, deadoponerrorinfunclet_r, and deadoponerrorinfunclet_d should be disabled for LLVM AOT; these are genuine failures.

[naricc]

Looks like hte libraries failures are this issue: #52031

[naricc]

Also this one: https://github.com/dotnet/core-eng/issues/12969

[naricc]

Some how I didn't merge this. Re-opening.

[SamMonoRT]

Don't see how your changes can cause Library/WASM lane failures @steveisok are these known issues and we can merge this PR ?