-
Notifications
You must be signed in to change notification settings - Fork 17.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
runtime: programs compiled by 1.11 allocate an unreasonable amount of virtual memory #28114
Comments
Related: #28081 (large core files) |
CC @aclements |
Presumably related to https://golang.org/cl/85888 and #10460. |
Interesting. I had thought that The solution I proposed in #28081 should also fix this. |
More discussion: https://news.ycombinator.com/item?id=18184541 |
Thanks for the hint @aclements. Indeed, couple of my system programs are using
As observed by host OS, it appears to have indeed copied all of the allocated virtual memory into main memory:
Hehe also I was wondering why this issue attracted so many emojis, it turns out someone put it on hackernews.. |
Yeah, I had just upgraded to 1.11 while not noticing anything significant in the release notes (at that time). And just after that, I opened hacker news, and this was the top result. Imagine the panic! (pun intended) |
@HouzuoGuo, yes, that's what I would expect. Thanks for confirming. I don't really understand when it makes sense to use Even if we do modify arena mapping to be incremental (which I believe should be relatively easy to do), I don't want to change how the arena index is mapped because that would impact both performance and simplicity. So |
Thanks @aclements , TIL about MCL_ONFAULT, apparently available since Linux 4.4. I shall use it from now and onward. Beyond the scope of this issue report, would you please offer some hints on the proper invocation of mlock in protecting a sensitive instance of structure, especially to determine its memory address range? Take this structure for example
|
Folding this issue into #28081, which I've re-titled, since the root cause is the same. |
What version of Go are you using (
go version
)?go1.11.1 linux/amd6
Does this issue reproduce with the latest release?
It was first discovered under go 1.11, and the issue remains in the minor upgrade to 1.11.1.
What operating system and processor architecture are you using (
go env
)?What did you do?
Compile the following trivial program:
go build -o usagetest; ./usagetest
and observe its memory usage as monitored by host operating system, for example viaps aux | grep usagetest
.What did you expect to see?
Compiled by 1.10.4, the program uses a very reasonable amount of virtual (2MB) and resident (600KB) memory:
howard 10838 101 0.0 2384 688 pts/2 Rl+ 05:57 0:59 ./usagetest
But when compiled by 1.11 (or 1.11.1), the virtual memory usage dramatically increases to 102MB:
howard 11326 99.5 0.1 101820 4776 pts/2 Rl+ 05:59 1:58 ./usagetest
The significant increase in virtual memory usage is usually not an issue, however security sensitive programs often lock their memory, causing a far greater performance degradation on low-spec computer hosts.
The text was updated successfully, but these errors were encountered: