VM require more memory

[marmarek]

Qubes OS version (e.g., R3.2): R4.0

Affected TemplateVMs (e.g., fedora-23, if applicable): fedora-25

---

When VM is booted as HVM, it very quickly get out of memory in default settings:

• sys-net with memory=maxmem=300MB is very slow (swapping)
• other VM with maxmem=4000MB, memory=400MB get oom during startup (fork: Cannot allocate memory on most services, including udev), this happens even before swap is enabled

Increasing memory setting do improve the situation, but also limit number of possible VMs.

[marmarek]

One thing is that SWIOTLB is enabled, even when no PCI device is assigned (this is Linux default when there is more than 3GB of RAM). This by default use 64MB of physical memory, which cannot be used for anything else. Reducing it to 16MB (default setting on Qubes 3.2) by swiotlb=8192 kernel option improve situation:

• sys-net use less swap (I'd say it is usable)
• other VM manage to reach the point where swap is enabled; after full startup about 50MB of swap is used

The results depends on maxmem setting because Linux kernel allocate metadata for all the memory, not only this really usable.

[marmarek]

One thing is that SWIOTLB is enabled, even when no PCI device is assigned (this is Linux default when there is more than 3GB of RAM). This by default use 64MB of physical memory, which cannot be used for anything else. Reducing it to 16MB (default setting on Qubes 3.2) by swiotlb=8192 kernel option improve situation:

• sys-net use less swap (I'd say it is usable)
• other VM manage to reach the point where swap is enabled; after full startup about 50MB of swap is used

The results depends on maxmem setting because Linux kernel allocate metadata for all the memory, not only this really usable.