Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: use MADV_FREE on linux as well #23687

Closed
CAFxX opened this issue Feb 4, 2018 · 3 comments

Comments

@CAFxX
Copy link
Contributor

commented Feb 4, 2018

On linux currently sysUnused uses madvise(MADV_DONTNEED) to signal the kernel that a range of allocated memory contains unneeded data. After a successful call, the range (but not the data it contained before the call to madvise) is still available but the first access to that range will unconditionally incur a page fault (needed to 0-fill the range).

A potentially faster alternative is MADV_FREE, available since linux 4.5 (and already used for bsd). The mechanism is very similar, but the page fault will only be incurred if the kernel, between the call to madvise and the first access, decides to reuse that memory for something else.

Off the top of my head, there are two ways to support both pre-4.5 and 4.5+ kernels:

  • Use a fallback approach (i.e. try MADV_FREE first and MADV_DONTNEED if MADV_FREE fails): this is e.g. what node does
  • Detect at runtime startup whether the running kernel supports MADV_FREE (e.g. by attempting to call madvise(MADV_FREE) on an allocated but unused range)
@bradfitz

This comment has been minimized.

Copy link
Member

commented Feb 4, 2018

@gopherbot

This comment has been minimized.

Copy link

commented Sep 14, 2018

Change https://golang.org/cl/135395 mentions this issue: runtime: use MADV_FREE on Linux if available

@alk

This comment has been minimized.

Copy link

commented May 28, 2019

Hi all. I wondering if there is any data backing up claims about MADV_FREE actually improving anything.

Asking because:

a) I have not seen any data pointing that it does help (I co-maintain gperftools and over there we're defaulting MADV_FREE to off on Linux).

b) there are numerous questions about implementation of this feature in kernel. First, while it is somewhat cheaper then MADV_DONTNEED, it is not that cheap, since page tables have to be walked and modified. It doesn't get reflected in RSS and thus interaction of this and memory reclaim facility in kernel, especially in settings with heavy usage of control groups isn't trivial. And also defaults about how this memory is preferred compared other memory during reclaim is also not entirely clear.

Since this feature was lobbied by jemalloc folks, maybe there are some Facebook folks involved in golang who do have some data?

I think this ticket should be re-opened. We should either gather and publish the data and see that indeed it helps, or revert defaults to off. IMHO.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.