Memory is not being freed on VZ

[okch-codes]

Discussed in #2720

^{Originally posted by Muzosh October 10, 2024}
I ran some memory intensive task today and it maxed out at 8GB (which was defined in my config). I was this in MacOS Activity Monitor next to a "Virtual Machine Service for limactl" process.

However, once those memory intensive tasks finished, the process stayed at 8GB indefinitely (until colima restart), slowing down my host Mac.

Is this intended behavior? I would expect the process to jump down to some idle value (around ~800-1000MB after fresh start).

[afbjorklund]

It is normal for memory that you dedicate to a virtual machine to be locked, the alternative is a feature*.

* called "ballooning" in QEMU, but I am not sure if it is available in Vz yet? Either way, it is not the default.

[okch-codes]

Apparently it's available but it doesn't work.

[balajiv113]

Yes its a macOS bug. Even docker desktop has the same problem.

Somehow orbstack fixed it with a custom memory management
https://orbstack.dev/blog/dynamic-memory

[afbjorklund]

Lima does create a VZVirtioTraditionalMemoryBalloonDeviceConfiguration, but I am not sure if it is used yet?

https://developer.apple.com/documentation/virtualization/vzvirtiotraditionalmemoryballoondevice

[micahthomas]

Reading the docs, looks like lima would have to do some work to make the VZVirtioTraditionalMemoryBalloonDevice work:

To use a memory balloon device, change the value in the targetVirtualMachineMemorySize property when your virtual machine is running. If the new value is smaller than the amount of currently assigned memory, the guest system may return a list of unused memory pages using the memory balloon device. If it does, the virtual machine releases those pages back to the host computer. If it doesn’t return any memory pages, the virtual machine leaves the guest’s memory size unchanged. If the new value is larger than the amount of currently assigned memory, the virtual machine reserves more pages for the guest operating system.

For optimal performance, the guest operating system should compact its memory before returning any pages back to the memory balloon device. Compacting the memory reduces fragmentation, and allows it to return contiguous blocks of free pages in the memory balloon device.

I am guessing that orbstack is periodically checking the memory stats, and if the usage is lower than vm allocation - lowers the value for the VM. Maybe there are some other things involved like asking linux to compact pages or other optimizations.

Would this make sense to implement in lima?