Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: rc1.14rc1 build internal/ssa out of memory #37122

Closed
n2vi opened this issue Feb 7, 2020 · 13 comments
Closed

cmd/compile: rc1.14rc1 build internal/ssa out of memory #37122

n2vi opened this issue Feb 7, 2020 · 13 comments
Milestone

Comments

@n2vi
Copy link

@n2vi n2vi commented Feb 7, 2020

What version of Go are you using (go version)?

go version go1.13.7 openbsd/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

See attached file.

What did you do?

git pull
git checkout go1.14rc1
cd src
./all.bash

What did you expect to see?

compile and test results as usual

What did you see instead?

Building go toolchain3 using go_bootstrap and Go toolchain2
# cmd/compile/internal/ssa
fatal error: runtime: out of memory

There was actually plenty of available RAM at the time, according to top.

go1.14rc1.txt

@cagedmantis cagedmantis modified the milestones: Backlog, Go1.14 Feb 7, 2020
@cagedmantis
Copy link
Contributor

@cagedmantis cagedmantis commented Feb 7, 2020

@cagedmantis cagedmantis changed the title rc1.14rc1 build internal/ssa out of memory cmd/compile: rc1.14rc1 build internal/ssa out of memory Feb 7, 2020
@josharian
Copy link
Contributor

@josharian josharian commented Feb 7, 2020

There was actually plenty of available RAM at the time, according to top.

What does "plenty" mean here? I don't know how quickly top refreshes, but building package ssa eats ram (and potentially dies) very quickly.

I suspect this is probably a duplicate of #27739. I have a series of CLs ready for 1.15 that will shrink package ssa by about 25%.

@n2vi
Copy link
Author

@n2vi n2vi commented Feb 7, 2020

@josharian
Copy link
Contributor

@josharian josharian commented Feb 7, 2020

Yeah, 12gb really ought to be enough. :) Does it reproduce consistently?

cc @mknyszek @aclements just in case

@n2vi
Copy link
Author

@n2vi n2vi commented Feb 7, 2020

Yes, reproduces 100%. Happy to re-run with any additional debugging you can teach me.

In any event, if a normal Go install requires tons of memory, it is probably worth measuring this at release candidate build time and adding a comment to the install doc, or even better finding a way to determine at runtime that the process is likely to fail for a known reason.

@n2vi
Copy link
Author

@n2vi n2vi commented Feb 7, 2020

Also, just reproduced it on another OpenBSD machine running stable+syspatch, so this is probably not some recent OpenBSD issue.

I'm happy to give Austin a login if constructing such a machine would be burdensome to y'all. I know I'm running on a niche OS and aspire to the skill to give you a better bug report.

@josharian
Copy link
Contributor

@josharian josharian commented Feb 7, 2020

To rule out some things, let's try disabling parallel and concurrent compilation:

GOFLAGS="-p=1 -gcflags=cmd/compile/internal/ssa=-c=1" ./all.bash

If that still fails, let's see how much RAM the shell thinks it is using:

GOFLAGS=-toolexec="/usr/bin/time -l" ./all.bash

@aclements
Copy link
Member

@aclements aclements commented Feb 7, 2020

OpenBSD by default has a very low RLIMIT_DATA. What's the output of ulimit?

Even if RLIMIT_DATA is raised, OpenBSD limits anonymous mappings to a fairly small factor larger than physical memory (even if they're untouched), and Go 1.14 does create some larger sparse mappings than Go 1.13 did. For this limit, the relevant top column to watch would be VSZ, I believe.

@aclements
Copy link
Member

@aclements aclements commented Feb 7, 2020

What's the output of ulimit?

Sorry, ulimit -a.

@n2vi
Copy link
Author

@n2vi n2vi commented Feb 7, 2020

"ulimit -a" on this 16GB physical NUQkit reports stack 8MB, lockedmem 5GB, mem 16GB.

GOFLAGS="-p=1 -gcflags=cmd/compile/internal/ssa=-c=1" ./all.bash
still fails in the same way.

Maybe I'm blind, but
GOFLAGS=-toolexec="/usr/bin/time -l" ./all.bash
seems to do the same with no extra output. Sorry, I don't know your build system enough to do better yet.

@josharian
Copy link
Contributor

@josharian josharian commented Feb 8, 2020

Sounds like Austin probably has a better bead on what’s going on here.

I’m curious though whether my planned changes will help sufficiently. To test,

git fetch "https://go.googlesource.com/go" refs/changes/03/213703/6 && git checkout FETCH_HEAD

and then try again.

@n2vi
Copy link
Author

@n2vi n2vi commented Feb 8, 2020

Hurray and thank you! With your change, builds and tests successfully.

@n2vi n2vi closed this Feb 8, 2020
@josharian
Copy link
Contributor

@josharian josharian commented Feb 8, 2020

That’s good to hear. That won’t land until the 1.15 release, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.