Officializing the Incremental GC

[luc-blaeser]

Officializing the Incremental GC

After a longer beta testing phase and some fine-tuning, we could officilalize the incremental garbage collector.
This would be next intermediate step before making the incremental GC the default GC in Motoko.

Some advice may be helpful to be given to developers, ideally also in a forum post, once we have declared the GC production ready.

Note

The incremental GC is enabled by the moc flag --incremental-gc (#3837) and is designed to scale for large program heap sizes.

While resolving scalability issues with regard to the instruction limit of the GC work, it is now possible to hit other scalability limits:

• Out of memory: A program can run out of memory if it fills the entire memory space with live objects.
• Upgrade limits: When using stable variables, the current mechanism of serialization and deserialization to and from stable memory can exceed the instruction limit or run out of memory.

Recommendations

• Test the upgrade: Thoroughly test the upgrade mechanism for different data volumes and heap sizes and conservatively determine the amount of stable data that is supported when upgrading the program.
• Monitor the heap size: Monitor the memory and heap size (Prim.rts_memory_size() and Prim.rts_heap_size()) of the application in production.
• Limit the heap size: Implement a custom limit in the application to keep the heap size and data volume below the scalability limit that has been determined during testing, in particular for the upgrade mechanism.
• Avoid large allocations per message: Avoid large allocations of 100 MB or more per message, but rather distribute larger allocations across multiple messages. Large allocations per message extend the duration of the GC increment. Moreover, memory pressure may occur because the GC has a higher reclamation latency than a classical stop-the-world collector.
• Consider a backup query function: Depending on the application case, it can be beneficial to offer an privileged query function to extract the critical canister state in several chunks. The runtime system maintains an extra memory reserve for query functions. Of course, such a function has to be implemented with a check that restricts it to authorized callers only. It is also important to test this function well.
• Last resort if memory would be full: Assuming the memory is full with objects that have shortly become garbage before the memory space has been exhausted, the canister owner or controllers can call the system-level function __motoko_gc_trigger() multiple times to run extra GC increments and complete a GC run, for collecting the latest garbage in a full heap. Up to 100 calls of this function may be needed to complete a GC run in a 4GB memory space. The GC keeps an specific memory reserve to be able to perform its work even if the application has exhausted the memory. Usually, this functionality is not needed in practice but is only useful in such exceptional cases.

[github-actions]

Comparing from 4271e7c to 57ff4db:
The produced WebAssembly code seems to be completely unchanged.

[crusso]

I wonder when we should make it the default?

[luc-blaeser]

I wonder when we should make it the default?

I thought that we could maybe wait for one more month to see if there is user feedback after this "blessing" and then make it the default.

[ggreif]

Maybe we should add a (GH) wiki page with the decision procedure for when not to use incremental GC, as well as a pros/cons table about each GC choice.

[rossberg]

Did the forum announcement just officially recommend the direct use of Prim functions? That seems like a very bad precedent. ;)

[luc-blaeser]

Did the forum announcement just officially recommend the direct use of Prim functions? That seems like a very bad precedent. ;)

Thanks for noticing. I agree. We should better offer some metrics in the base library. I will prepare such change and then update the post.