Optimize generic MethodInfo for Func<TResult>

[sharwell]

Repeated profiling showed MarshalInvoke as a heavy performance bottleneck. Rather than remove the use of reflection at runtime altogether, this pull request demonstrates a strategy for caching the results of particularly expensive operations. The performance benefits come from two primary changes:

1. RuntimeReflectionExtensions.GetMethodInfo(Delegate) is only called once for a given method
2. MethodInfo.GetGenericMethodDefinition() is only called once for a given method

In this change, I am also caching the result of MethodInfo.MakeGenericMethod. It might not be necessary to cache this result, but it doesn't seem to hurt either.

📝 This pull request is a demonstration of the process required to fully convert the code from the current pattern to the new pattern. I have only converted MarshalInvoke<TRet>; each of the others will need to be converted in a similar fashion.

[codemzs]

@sharwell This seems like a promising change but can we please get before and after performance numbers and speed improvements on typical ML pipelines?

[sharwell]

@codemzs The only number I will be able to get is the total allocation costs prior to my change in the context of running tests. I'll run a measurement on Microsoft.ML.Tests and show the total allocations within the MarshalInvoke methods and the percentage of those relative to the whole run.

[sharwell]

Due to data collection limitations, I was only able to capture allocations for 11 minutes 20 seconds in the middle of the test run. Total allocations during the recorded time were 143.9GB. A lower bound of allocations directly attributable to the MarshalInvoke pattern (and would be resolved by the pattern seen here) were 44GB.

[harishsk]

Any idea why the file names of the newly added files in this directory have an backtick-1 or backtick-2 suffix? Is that an accident or is there a specific reason behind this naming? The change looks good and I am approving it but it would be good if the naming can be fixed.

[sharwell]

The file names match the names of the types defined in the files. When generic types are compiled, the compiler adds a `{arity} suffix to the type name.

[harishsk]

What does the {arity} represent here?

---

In reply to: 386548523 [](ancestors = 386548523)

[sharwell]

The number of generic type parameters.

[sharwell]

I'm using the term from here:
https://github.com/dotnet/roslyn/blob/6389e2519f282f7683025761189894e1c894936c/src/Compilers/Core/Portable/Symbols/INamedTypeSymbol.cs#L23-L27

I wanted to link to the language specification but it seems the specification doesn't explicitly give a name to this characteristic of generic types.

[harishsk]

Thanks for the additional info. Please feel free to merge.

---

In reply to: 386566851 [](ancestors = 386566851)

[sharwell]

I'll run a build locally to make sure nothing changed in the interim and merge if it passes.

[harishsk]