Add support for dehydrated runtime data structures

[MichalStrehovsky]

This adds support for dehydrating pointer-rich data structures at compile time and rehydrating them at runtime.

NativeAOT compiler generates several pointer-heavy data structures (the worst offender being MethodTable). These data structures get emitted at compile time and used at runtime to e.g. support casting or virtual method dispatch.

We want to be able to generate structures that have pointers in them because e.g. virtual method dispatch needs to be fast and we don't want to be doing extra math to compute destination (just dereference a pointer in the data structure the compiler generated and call it).

But pointers are big, and there's extra metadata the OS needs in the executable file on top of that (2 bytes on Windows, 24 (!!) bytes on Linux/ELF).

This adds support for "dehydrating" the data structures with pointers at compile time (representing pointers more efficiently) and "rehydrating" them at runtime.

The rehydration is quite fast - I'm seeing 2.2 GB/s throughput on my machine. Hello world rehydrates under a millisecond.

The size savings are significant: 7+% on Windows, 30+% on Linux.

Depends on #77972 getting through (with the update from llvm-project repo).

Cc @dotnet/ilc-contrib

[VSadov]

So far I assumed we could have method table pointers emitted in the native code and relocations would take care of them when loaded. But how that would work with rehydration of the method tables?
For example if I have if (o.GetType() == typeof(string)) , would I have an address to string's method table baked into the native code? What happens when I new something? How the code knows where the rehydrated method table is located? Or is the rehydration done in-place? (like we reserve the space and store enough data to later rehydrate/patch up). How do we guarantee the data fits (i.e. compression rate is better than 1)?

I am obviously missing some important parts. I think some more explanation on how this works could be helpful.

[MichalStrehovsky]

The comments are somewhat scattered across the code, but the key part is:

                        // If the object node is getting dehydrated, emit it into a zero-initialized
                        // data section along with all its symbols.
                        // Dehydrated data will be emitted elsewhere.

So the executable now defines an extra region within the .bss section. Code points to this region. It is zero filled at startup by the OS and we "decompress" the compressed data structures into it during very early startup. At runtime there's no difference in efficiency of accessing this area - it's still all direct pointers - the only difference is that previously it would be a pointer to the "const data" section, and after this it would be a pointer to "uninitialized data" that we initialized ourselves.

[VSadov]

Code points to this region. It is zero filled at startup by the OS and we "decompress" the compressed data structures into it during very early startup.

I see. Makes sense.
Alternatively, it could be just writeable data filled with compressed representation + padding to reserve correct size, but then we would need to be sure compressed representation actually fits and the obj file would need to store the padding.

[danmoseley]

this looks like IfFailGo type macro style. Why not just throw onerror instead of error codes?

[MichalStrehovsky]

This is under #if false - it was my debugging helper to make sure I didn't mess up the bit packing (this file can be compiled on its own with the region enabled).

I'm still debating whether to check it in, delete it, or spend more time converting it to a unit test that is never going to fail because nobody will want to touch this anyway.

[sbomer]

Neat! LGTM as someone not super familiar with the code. :)

[LakshanF]

Thanks for walking through the code and doing this work to reduce the size!

I took a look at the Encode and Decode routines on DehydratedDataCommand class and the testing there. The testing seems to cover the 1 byte boundary for EncodeShort well and I couldn't think of any missing cases. I also took look at the DehydratedDataNode & StartupCodeHelpers where this is used and while not fully able to follow the pointer manipulation for RehydrateData for the last 2 options, I assume the lookup table logic holds (and will blow up the program if it doesn't :-))

Hopefully the dependent update merge happens soon so the larger testing matrix can follow.

[VSadov]

Buffer.MemoryCopy ?

[MichalStrehovsky]

We can't call that from Test.CoreLib. I'm also a little bit worried - this code runs during very early startup - casting, typeof, newobj and many other things are not available. Buffer.MemoryCopy feels a bit higher up the stack that I'm not sure it would be safe to call.

[VSadov]

Yes this runs very early. I am not sure if Buffer.MemoryCopy does anything complex, but since the data here is unmanaged, it would eventually just call something like InternalCalls.memmove.

Are these copied chunks long enough to involve memmove? If they are typically just 10-20 bytes, it may not get any faster.

[VSadov]

LGTM! Nice!

[lambdageek]

/cc @ivanpovazan we should rerun our sample after this is merged and see if we get more size reduction

[eerhardt]

I can confirm this change makes a major size reduction on Linux. Publishing NativeAOT a simple ASP.NET WebAPI app:

Before: 29.31 MB
After: 19.78 MB

This is a 32.5% size reduction!

[MichalStrehovsky]

I can confirm this change makes a major size reduction on Linux. Publishing NativeAOT a simple ASP.NET WebAPI app:

Nice! Thank you for measuring it! #78545 should bring another maybe ~1% saving, and I have a list of a couple more savings (not quite 32.5%, but maybe another 5%).

[eerhardt]

Just a quick clarification - I later discovered that these numbers also included #78198, which cut ~1 MB of that app size off. So the full 32.5% decrease was with both of these changes. #77884 contributed the vast majority of it though.