OOM with Excessive Memory Usage During Git Repository Import

[genedna]

I am writing to report an issue I encountered while using Mega to import a Git repository. The repository in question is https://github.com/AFLplusplus/qemu-libafl-bridge.

During the import process, I noticed that Mega was consuming excessive memory while parsing Pack files. This high memory usage ultimately triggered the Linux system's Out-Of-Memory (OOM) killer, resulting in the termination of the Mega process.

So that you know, the system I am using has 16GB of RAM. Under idle conditions, it typically has around 13GB of available memory. Before this incident, no specific optimizations were applied to the system or Mega settings.

[genedna]

Upon analysis, it seems that the system, not having a swap partition, resorts to invoking the OOM killer when faced with high memory usage, leading to the termination of running programs. If sufficient memory cannot be freed, the system kills the active processes.

The current hypothesis is that this issue stems from poor memory control management during decoding Pack files. As a result, I am planning to rewrite this portion of the code to handle memory usage during decoding better.

[genedna]

I want to thank @Ivanbeethoven sincerely for their significant contribution to the project. Specifically, separating the Delta algorithm from the Pack encoding/decoding process is a remarkable improvement, as accomplished in pull requests #246 and #249.

The separation of concerns achieved through this refactoring not only enhances the readability and maintainability of the code but also opens up possibilities for reuse and extension in other areas of the project. I am excited about the potential application of this approach in the 'MDA' project in the feature through #202.