-
Notifications
You must be signed in to change notification settings - Fork 18.4k
Description
Because the Go runtime's background scavenger releases memory in 64 KiB chunks, the heuristic in mem_linux.go
's implementation of sysUnusedOS
never actually calls MADV_NOHUGEPAGE
to break up huge pages that would otherwise keep a lot of memory around. The reason for this is that there's a heuristic preventing the call on every sysUnusedOS
to avoid creating too many VMAs (Linux has a low limit).
The result of all this is that Linux may be keeping around huge pages transparently using up a lot more memory than intended, and reducing the effectiveness of the scavenger.
I think the fix for this is to mark all new heap memory as MADV_NOHUGEPAGE
initially, and then only call it again in circumstances where we free memory that is likely to have had MADV_HUGEPAGE
called on it (e.g. a large object that contains at least one aligned 2 MiB region will have this happen via sysUsed
). However, this might not be enough, and needs some more investigation.
CC @golang/runtime