Proposal: Application State Checkpoint/Restore (Snapshotting) for JIT Warmup

[Mortezamir81]

Background

The .NET team has done an incredible job addressing cold start times, primarily through Native AOT. While Native AOT is fantastic, it comes with strict limitations: lack of dynamic code generation (Reflection.Emit), constrained reflection, and ecosystem incompatibility.

For many large, legacy, or highly dynamic enterprise applications, moving to Native AOT is simply not possible. These apps still suffer from long startup times and high CPU spikes during initial JIT compilation, which is a major pain point in Serverless (AWS Lambda, Azure Functions) and auto-scaling Kubernetes environments.

Proposal

I would like to propose adding support for Application Checkpoint/Restore, similar to the CRaC (Coordinated Restore at Checkpoint) API in the JVM ecosystem or Node.js snapshotting.

The workflow would look like this:

1. Warmup Phase (Build/Deploy time): The application is started. A script drives mock traffic to it. The JIT compiles the hot paths, and caches/DI containers are populated.
2. Checkpoint: The runtime takes a snapshot of the entire memory state (the heap, JITted code, etc.) and dumps it to disk.
3. Restore Phase (Production): When a new instance is needed, the runtime boots directly from the snapshot.

Value Proposition

• The Best of Both Worlds: We get sub-millisecond, AOT-like startup times without sacrificing peak JIT performance (Tiered Compilation, PGO) or breaking Reflection/dynamic capabilities.
• Serverless Dominance: This would make standard JIT-compiled C# an absolute powerhouse in serverless environments, removing the cold start penalty entirely.

Currently, implementing something like CRIU (Checkpoint/Restore In Userspace) on Linux with .NET is highly unstable and unsupported. Having native, coordinated runtime support (where the framework knows how to pause background threads, close file handles, and re-open network sockets on restore) would be a massive leap forward for .NET backend development.

[dotnet-policy-service]

Tagging subscribers to this area: @agocke, @dotnet/ilc-contrib
See info in area-owners.md if you want to be subscribed.

[Mortezamir81]

Anticipated Challenges & Mitigations

Implementing Checkpoint/Restore at the runtime level carries known challenges, but the JVM/CRaC ecosystem has proven they are solvable:

1. RNG Entropy & Security: When the runtime is restored, it must automatically re-seed the global random number generators (Random.Shared and Cryptographic providers) by pulling fresh entropy from the host OS (e.g., /dev/urandom) to prevent duplicate tokens across scaled instances.
2. CPU Instruction Set Mismatch: To prevent crashing when restoring on a different CPU architecture in the cloud, the warmup phase could be configured to JIT-compile using a baseline instruction set (e.g., forcing maximum AVX2, ignoring AVX-512 during the checkpoint phase).
3. Open Handles/Sockets: The framework could introduce lifecycle events (e.g., ApplicationLifecycle.OnCheckpointing and ApplicationLifecycle.OnRestoring). Libraries like Kestrel, HttpClientFactory, and EF Core would implement these to cleanly drain and close connections before the snapshot, and re-initialize them upon restore.

[KalleOlaviNiemitalo]

How much would this weaken ASLR?

[Mortezamir81]

That is a very valid security concern. You are correct that restoring from a fixed snapshot effectively "freezes" the address space layout, which negates the entropy provided by ASLR across multiple restored instances.

However, this is a known trade-off in the Checkpoint/Restore ecosystem (similar to what's discussed in CRIU and Java's CRaC). A few ways we could address or mitigate this:

1. Deployment Context: This feature is primarily aimed at high-performance, short-lived workloads (like Serverless or Microservices) where the speed of startup is the priority.
2. Partial Re-randomization: Some advanced restore mechanisms attempt to re-map certain memory regions (like the stack or specific heap segments) upon restore, though this is architecturally complex for the JIT.
3. Opt-in Risk: Like many performance-oriented features, this would be strictly opt-in. Users who require maximum ASLR entropy for every single instance might choose not to use snapshotting, while those needing sub-50ms startup for cold-starts may accept the trade-off.
4. Coordinated Lifecycle: Using the proposed OnRestoring hooks, an application could theoretically sensitive data or rotate keys to ensure that even if the layout is static, the application's internal state remains secure.

I'd love to hear if the JIT team has explored "ASLR-aware" snapshots before!

@KalleOlaviNiemitalo

[jkotas]

We have looked into this before. ASLR is not the only concern. Any secrets and random seeds are a problem. The security profile of this feature is very challenging, and it does not meet out security bar.

"ASLR-aware" snapshots before

It is effectively ReadyToRun compilation. PublishReadyToRun allows you precompile the application code at build time while preserving ASLR.