[mono] Investigate rewriting Scalar<T> methods so the linker can replace them with stubs

[ivanpovazan]

Scalar is defined with struct constraint, but additionally in many places (in different methods) there are runtime checks to check for a specific type support. If the type is not supported the method just throws.
AOTing such pattern with Mono results in unnecessary large and slow methods.

Rewriting methods of Scalar in a way that linker can replace them with stubs can potentially reduce the size of the AOTed code by Mono.

This is related to #56385

cc @vargaz

[tannergooding]

The pattern that Scalar<T> follows is a fairly standard pattern throughout the .NET ecosystem, particularly in high performance code.

I think it would likely be better, long term, for Mono to adjust itself to better handle the pattern.

[ivanpovazan]

hi @tannergooding, the idea was to create a tracking issue for investigating whether it is possible to rewrite the Scalar<T> so its code could get trimmed down (especially since it is, as far as I understand, used only internally), which would help mono AOT to produce smaller binaries.
Maybe I should have labeled it differently since it is an investigation rather than a problem (please let me know if you want me to change the labels).

On the other hand, yes, you are right, there is a more concrete solution tracked here: #71431 while this one is a faster approach to see the potential gains.

[tannergooding]

Maybe I should have labeled it differently since it is an investigation rather than a problem (please let me know if you want me to change the labels).

I think the confusing part is simply that it says Rewrite rather than something like Investigate rewriting.

Scalar<T> is largely a helper type meant for platforms without SIMD acceleration and as a fallback for reflection, delegates, or other forms of "indirect invocation" (even on platforms with SIMD acceleration).

We could make it smaller for platforms with SIMD acceleration by making them recursive on one path (much like the Hardware Intrinsics do), that way the Scalar<T> path is "dead code" except for platforms like Arm32.

But otherwise, I don't expect there is a simplification, especially not one that is expressible in C#. I don't think representing the fallback path in the JIT is a good idea as it defeats the purpose of having a fallback and requires "more bringup" work for new or community supported platforms/architectures.

[huoyaoyuan]

Where is the opportunity to rewrite such methods with generic math? Such pattern is often used for high performance code.
The current generic math type system lacks a way to represent the check around constraint. Further improvement should take this in mind.

[tannergooding]

Generic Math isn't possible for this scenario right now because there is no way to go from something "less constrained" to something "more constrained"

That is, given where T : struct, you can't call something where T : struct, INumber<T> because the language and runtime have no way to make this a valid check, etc.

Supporting this scenario in the future has been raised and will likely be taken into consideration

[lambdageek]

I think it would likely be better, long term, for Mono to adjust itself to better handle the pattern.

The problem with this pattern is that it causes us to generate a ton of fallback code for cases that can't happen. For any concrete instantiation with a supported type the code is fine: the typeswitch compiles down to a single branch and the overall function is usually small enough to get inlined into the caller, etc.

But we also have to generate a universal generic fallback. This thing by definition doesn't know anything about the generic type so it has to generate a huge type switch and use the slower gsharedvt (universal generics / boxing / dictionary passing) approach.

There are principled things we can do (especially if aided by hints to the AOT compiler about which specializations are profitable to generate), but getting there will take some engineering effort. In a closed universe like an iOS application, rewriting with the IL trimmer may give us an indication of the potential space savings if nothing else.

[tannergooding]

Sure, but my point is that this is a very common pattern in perf oriented code.

Within corelib its used by vector code, its used by generic math, its also used in various other hot paths to ensure specialization occurs. Outside the ecosystem its used by all sorts of libraries and applications.

So this is something that will ultimately have to be handled or it will continue presenting as a negative experience for Mono. Not just here, but throughout the entire ecosystem.

---

This thing by definition doesn't know anything about the generic type so it has to generate a huge type switch and use the slower gsharedvt (universal generics / boxing / dictionary passing) approach.

It seems like, at the very least you could recognize the if (typeof(T) == typeof(...)) pattern. For sealed types, like value types, this is definitively "live" or "dead". It's only for unsealed reference types where that can be truly unknown, but that scenario is also increasingly uncommon since RyuJIT only specializes on value types.

It would then seem reasonable to assume that with such a pattern, the specializations to consider are exactly the typeof(...) checks; most of which could be trimmed in a real application since they are unused.

In the case of things like Scalar<T> one of the paths (the fallback path) always throws and so the compiler should also be able to recognize this and keep that path as the "shared" path.

[jkotas]

It would then seem reasonable to assume that with such a pattern, the specializations to consider are exactly the typeof(...) checks; most of which could be trimmed in a real application since they are unused.

It is typically impossible to tell whether these specializations are unused thanks to reflection.

[tannergooding]

We have attributes and other annotations designed to assist with that. I don't think its unreasonable to require or expect those to be used here as well.

These types of patterns aren't going to go away. In the worst case the AOT codegen for this pattern could be n concrete (12 in this case) implementations and a central shared implementation that does the check and dispatch to the concrete implementation. Given the pattern, it would in general be no larger than a single shared implementation that contains all the paths, particularly given the generic on generic calls.

There are always exceptions, yes, but those tend to be rarer especially for this specific pattern.

[jkotas]

The existing trimming annotations do not track exact generic instantiations. They only allow you to say that MyType<T> is required, for unknown set of Ts.

These types of patterns aren't going to go away.

The problems that these patterns are trying to solve are not going away. I have doubts that the current popular patterns are the right way to solve these problems. These patterns are bad for startup, AOT or trimming. They are good only if your only performance metric is steady state RPS.