Long (and sometimes failing) native compilation with .NET native compiler

[velocitysystems]

Bug

When System.Reactive is used inside a Universal Windows Platform (UWP) application built in Release mode with .NET native (DNN) compilation enabled, native compilation time is extremely long to the point where it sometimes fails with out-of-memory errors thrown by the ILC compiler.

It has been suggested in #899 to add the property <Use64BitCompiler>true</Use64BitCompiler> to the UWP project, however this does not guarantee success.

Even with this property added to our application, our Windows store app which successfully built with System.Reactive 4.0 no longer compiles with .NET native when using System.Reactive 4.1 or higher. We are building on a powerful CI/CD pipeline with 16GB RAM and the ILC compiler often fails with out-of-memory errors such as:

Microsoft.NetNative.targets(801,5): error : Internal compiler error: Exception of type 'System.OutOfMemoryException' was thrown.

Which library version?

System.Reactive 4.1 and later.

What are the platform(s), environment(s) and related component version(s)?

Universal Windows Platform (UWP) application.

What is the use case or problem?

.NET native compilation takes a long time and often fails with out-of-memory errors.

What is the expected outcome?

.NET native compilation is successful and fast, or at least as fast as when System.Reactive 4.0 is used.

What is the actual outcome?

.NET native compilation is either:

• Many magnitudes slower to complete
• Fails entirely with out-of-memory errors

Do you have a code snippet or project that reproduces the problem?

Theory:
System.Reactive 4.1.x and greater causes the .NET native compiler to attempt to generate a huge amount of code, leading to long compile times.

Test:
NET Native Benchmarks.zip

The attached zip contains four UWP sample projects based on the latest blank app template.

Each sample uses a simple Observable.Timer on a 30s delay to log a timestamp to the console. The implementation is kept deliberately simple to provide an effective benchmark for comparing the .NET native compilation time between System.Reactive package versions.

Results:

• System.Reactive 4.0.0 (DotNetNativeSystemRX40) >= 30s to compile
• System.Reactive 4.1.6 (DotNetNativeSystemRX41) >= 1m 30s to compile
• System.Reactive 4.2.0 (DotNetNativeSystemRX42) >= 1m 30s to compile
• System.Reactive 4.3.1 (DotNetNativeSystemRX43) >= 1m 30s to compile

Analysis:

Thanks to @MattWhilden for the documentation on understanding the output of the ILC compiler.

One of the most helpful things for my day to day is understanding where the compiler puts various artifacts related to it’s mechanations. For the most part, these end up under obj<arch>\Release\ilc\intermediate.`

Each of the folders/items here is interestering in it’s own right but the ILTransformed directory is often the most helpful for issues like this. This directory contains most of the juicy artifacts from the “analysis” phase of compilation. Most importantly you’ll see:

*.csv files with general information
A closure file containing a list of all the dlls your app references
.csv with pdb and assembly info
A file to track all forwarded types
A file showing the result of all the decisions about which types need to be saved/generated etc due to reflection analysis
*.ildll files which are app dlls we’ve done various transformations to (still openable by ILSPY)
*.ilpdb files that help us write the final native pdb files

The most useful file in all of this is the Reflection log. It’s somewhat helpful to understand how the compiler thinks about reflection visibility etc.

I recommend glancing through this page to give yourself a sense of the things it’s trying to say. In particular, things marked as anything but Browse will cause the later phases of the compiler to need to do heavy lifting.

Looking at the size of the reflog for System.Reactive 4.3.1, the problem is clearly seen.

Diffing the two logs shows a huge number of members from the System.Reactive.Concurrency namespace now being marked as necessary. They also seem to all be marked as Dynamic which will cause the native code generator to need to kick in for each of them.

Possible Solution:
Add runtime directives into the System.Reactive package to instruct the .NET native compiler which types will never be dynamically invoked with reflection.

[clairernovotny]

This is fixed now in 4.2.2 and 4.3.2.

[velocitysystems]

Thanks @onovotny, this great news!
We appreciate the hard work.