♻ KeyCache-Garbage Collector

[CalebSerafin]

What type of PR is this.

• Bug
• Change
• Enhancement
• Library Additions

What have you changed and why?

• Added Garbage Collection Component of KeyCache.

But What is KeyCache???

• KeyCache's goal is to reduce data sent over a network.
• It can achieve this by replacing commonly used identifiers with a shortID.
• However, this technology can be used to create translation tables for any data.
• Translations can go stale and expire, this is preferable for a DNS-like implementation where the server holds all translations, and the client only needs to use a translation for a short period.
• KeyCache is intended to be used as middleware before and after remoteExecution. Therefore, it cannot be directly called from another machine. Only the server can directly update translations on clients.
• The parent code must handle the propagation of translations.
• The most critical part of expiring translations is the garbage collector. Therefore, this PR will add that component.

Tell us about this Garbage Collector

• This garbage collector finds stale translations and deletes them.
• A concurrent generational garbage collector was implemented.
• Generational GCs are based on life's tough truth that most objects die young. See https://docs.microsoft.com/en-us/dotnet/standard/garbage-collection/fundamentals#generations for a description.
• All new items are added to generation 0. If it not stale on this pass, it is promoted to generation 1.
• Generation 1 checks items at a slower pace. If items survive this pass, they are promoted to generation 2.
• Generation 2 checks items at an even slower pace. Items are not promoted past this point.

Please verify the following and ensure all checks are completed.

• Have you loaded the mission in LAN host?
• Have you loaded the mission on a dedicated server?

Is further testing or are further changes required?

1. No

• Yes

What do I need to review?

• You don't need to review the fields under /Tests/
• All functions that have content need a review
• CfgRemoteExec can use a review as well.

How can the changes be tested?

Steps:

/*
Run in order.
Make sure that the previous test finishes before starting next.
If the previous test failed, it's likely that the following will too.
*/

// Recommended
call compileScript ["functions\Utility\KeyCache\Tests\unitTest_garbageCollector_basicPromotion.sqf"];
call compileScript ["functions\Utility\KeyCache\Tests\unitTest_garbageCollector_timedPromotion.sqf"];
call compileScript ["functions\Utility\KeyCache\Tests\unitTest_garbageCollector_basicDeletion.sqf"];
// Will take ≈10 minutes. Game will still run at 60fps.
call compileScript ["functions\Utility\KeyCache\Tests\unitTest_garbageCollector_shortFpsStressTest.sqf"];

[CalebSerafin]

Long Stress Test Data

69 Million items were registered for GC. As they were being added, an equilibrium of ≈300 thousand soon-to-be GCed items formed.

Note: Arrays can't handle more than 10 million, and integers in scalars can't handle more than 16 million.

The GC uses buckets per generation to avoid array overfilling.
Raw data is available to the Offical Dev Team on its Discord.

The naive analysis might be that GC caused those lag-dips.
However, the GC is constantly scanning, promoting, and deleting items as noted by the equilibrium.
The total amount of non-deleted objects becomes reduced proportionally to the frame rate.
Therefore, the reduced framerate slowed down the stress-test script.
Therefore, It can be concluded that the GC outperformed the stress-test script.

[killerswin2]

5 empty files? Are they used in later versions, needed for something else, or holdover files?

[killerswin2]

Noice

[HakonRydland]

mostly good, dont really like the empty functions, should be added rn or when you actually want to implement the functionality

[CalebSerafin]

The function files have been put there and registered to functions.hpp and CfgRemoteExec.hpp in one swoop. Next change they will be filled.