Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: TestPageAllocScavenge/AllFreeUnscavExhaust fails on OpenBSD with 2.5GB RAM #36210

Closed
mpx opened this issue Dec 19, 2019 · 12 comments
Closed
Assignees
Milestone

Comments

@mpx
Copy link
Contributor

@mpx mpx commented Dec 19, 2019

go env Output
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/buildsrv/.cache/go-build"
GOENV="/home/buildsrv/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="openbsd"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="openbsd"
GOPATH="/home/buildsrv"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/buildsrv/go1.14beta1"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/buildsrv/go1.14beta1/pkg/tool/openbsd_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build974924072=/tmp/go-build -gno-record-gcc-switches"

Compiling go1.14beta1 via all.bash on OpenBSD 6.6 (amd64, 2.5GB RAM, 4 procs, no swap) fails. This can be reproduced with:

$ cd src/runtime
$ go1.14beta1 test -v -run TestPageAllocScavenge                         
=== RUN   TestPageAllocScavenge                                                                          
=== RUN   TestPageAllocScavenge/AllFreeUnscavExhaust                                                     
fatal error: failed to reserve page summary memory                                                                                                                                                                
goroutine 7 [running]:                                                                                   
runtime.throw(0x68cc1b, 0x25)                                                                                    /home/buildsrv/go1.14beta1/src/runtime/panic.go:1112 +0x72 fp=0xc00002e4e0 sp=0xc00002e4b0 pc=0x4
35cc2                                                                                                    
runtime.(*pageAlloc).sysInit(0xc0000c8000)                                                                       /home/buildsrv/go1.14beta1/src/runtime/mpagealloc_64bit.go:80 +0x13f fp=0xc00002e558 sp=0xc00002e
4e0 pc=0x42beef                                                                                          
runtime.(*pageAlloc).init(0xc0000c8000, 0xc00001a5a8, 0x0)                                               
        /home/buildsrv/go1.14beta1/src/runtime/mpagealloc.go:289 +0x75 fp=0xc00002e580 sp=0xc00002e558 pc
=0x429b85             
runtime.NewPageAlloc(0xc00006c7e0, 0xc00006c840, 0xc0000b6000)                                           
        /home/buildsrv/go1.14beta1/src/runtime/export_test.go:788 +0x70 fp=0xc00002e680 sp=0xc00002e580 p
c=0x463960
runtime_test.TestPageAllocScavenge.func1(0xc0000159e0, 0x3f9bae7f4000)
        /home/buildsrv/go1.14beta1/src/runtime/mgcscavenge_test.go:407 +0x124 fp=0xc00002e760 sp=0xc00002
e680 pc=0x5e9574
runtime_test.TestPageAllocScavenge.func2(0xc0000159e0)
        /home/buildsrv/go1.14beta1/src/runtime/mgcscavenge_test.go:413 +0x34 fp=0xc00002e780 sp=0xc00002e
760 pc=0x5e9744
testing.tRunner(0xc0000159e0, 0xc000043560)
        /home/buildsrv/go1.14beta1/src/testing/testing.go:954 +0xdc fp=0xc00002e7d0 sp=0xc00002e780 pc=0x
4de9fc
runtime.goexit()
        /home/buildsrv/go1.14beta1/src/runtime/asm_amd64.s:1375 +0x1 fp=0xc00002e7d8 sp=0xc00002e7d0 pc=0
x46ac31
created by testing.(*T).Run
        /home/buildsrv/go1.14beta1/src/testing/testing.go:1005 +0x357

[..] 
exit status 2
FAIL    runtime 0.116s

go1.13.5 builds fine with 2GB RAM.

I added some trace writes found mem_bsd.go:sysReserve mmap fails with ENOMEM trying to allocate 536870912 bytes (levelShift[4] in mpagealloc_64bit.go:sysInit).

Cc @mknyszek

@mknyszek

This comment has been minimized.

Copy link
Contributor

@mknyszek mknyszek commented Dec 19, 2019

This is all WAI as of Go 1.14 (see #35568). sysReserve is not allocation, only a reservation (the memory is PROT_NONE and untouched). The tests you're running here are end-to-end and create the same reservations the runtime does. The tests only ever make at 2 such reservations. When you include the reservations made by the currently executing runtime, you get a total virtual memory footprint of ~1.8 GiB to run these tests.

From #35568 my understanding is that even if you have datasize-max=infinity set, OpenBSD still imposes a per-process virtual memory ceiling based on the amount of physical memory available.

I'd like to avoid disabling these tests on OpenBSD, but perhaps it makes sense to disable them for non-builders so folks can run all.bash locally with the virtual memory requirements of one runtime, not 3.

@aclements @bradfitz WDYT?

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Dec 19, 2019

perhaps it makes sense to disable them for non-builders so folks can run all.bash locally with the virtual memory requirements of one runtime, not 3.

Note that all.bash sets -short, which should already skip memory-intensive tests. (Is this test missing an if testing.Short() check?)

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Dec 19, 2019

See also #33986, which discusses the need for a documented minimum RAM requirement for x/tools tests.

@mknyszek

This comment has been minimized.

Copy link
Contributor

@mknyszek mknyszek commented Dec 19, 2019

@bcmills the requirement is only virtual memory, not RAM. We only reserve the pages as PROT_NONE and map in a very small number (somewhere around 10 pages in the worst case). This is not a problem on non-OpenBSD systems. The physical memory footprint of each test execution, for the duration of that execution, is (IIRC) < 100 KiB.

That's the tricky bit. This only qualifies as "intensive" on OpenBSD (and maybe AIX? though I haven't seen any problems there after we made some fixes).

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Dec 19, 2019

Maybe use a combination of runtime.GOOS and testing.Short(), then?

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Dec 19, 2019

(And I'm assuming that the PROT_NONE makes the test equally inexpensive on Linux systems that have overcommit disabled?)

@mknyszek

This comment has been minimized.

Copy link
Contributor

@mknyszek mknyszek commented Dec 19, 2019

@bcmills OK, I'll put up a change for that.

Linux's overcommit rules in all settings (even the very strict ones!) are totally OK with large PROT_NONE mappings that haven't been touched before AFAICT.

@mknyszek mknyszek self-assigned this Dec 19, 2019
@mknyszek mknyszek added this to the Go1.14 milestone Dec 19, 2019
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Dec 19, 2019

Change https://golang.org/cl/212177 mentions this issue: runtime: disable pageAlloc tests on OpenBSD in short mode

@mknyszek

This comment has been minimized.

Copy link
Contributor

@mknyszek mknyszek commented Dec 19, 2019

@mpx could you please verify that https://golang.org/cl/212177 lets you run all.bash on your OpenBSD box?

@mpx

This comment has been minimized.

Copy link
Contributor Author

@mpx mpx commented Dec 20, 2019

TestPageCacheFlush also fails with:
fatal error: failed to reserve page summary memory.

Adding a Skip to TestPageCacheFlush allows the runtime tests to complete with -short.

Skipping the tests helps, but there is also a noticeable increase in the actual memory requirements for building the compiler under go1.14beta1 with OpenBSD. 2GB memory continues to work for FreeBSD, but not OpenBSD:

$ ./all.bash
Building Go cmd/dist using /home/buildsrv/go1.13.5. (go1.13.5 openbsd/amd64)
Building Go toolchain1 using /home/buildsrv/go1.13.5.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
# cmd/compile/internal/ssa
fatal error: runtime: out of memory
Stack trace
runtime stack:
runtime.throw(0xe53ea3, 0x16)
        /home/buildsrv/go1.14beta1/src/runtime/panic.go:1112 +0x72
runtime.sysMap(0xc040000000, 0x4000000, 0x1612158)
        /home/buildsrv/go1.14beta1/src/runtime/mem_bsd.go:73 +0xc5
runtime.(*mheap).sysAlloc(0x15fbe60, 0x400000, 0x20807afff, 0x1c00)
        /home/buildsrv/go1.14beta1/src/runtime/malloc.go:713 +0x1cd
runtime.(*mheap).grow(0x15fbe60, 0x1, 0x0)
        /home/buildsrv/go1.14beta1/src/runtime/mheap.go:1282 +0x11c
runtime.(*mheap).allocSpan(0x15fbe60, 0x1, 0xc0003d1000, 0x1612168, 0x20b7c94e8)
        /home/buildsrv/go1.14beta1/src/runtime/mheap.go:1120 +0x65d
runtime.(*mheap).alloc.func1()
        /home/buildsrv/go1.14beta1/src/runtime/mheap.go:867 +0x64
runtime.systemstack(0x45fd24)
        /home/buildsrv/go1.14beta1/src/runtime/asm_amd64.s:370 +0x66
runtime.mstart()
        /home/buildsrv/go1.14beta1/src/runtime/proc.go:1077

goroutine 113 [running]:
runtime.systemstack_switch()
/home/buildsrv/go1.14beta1/src/runtime/asm_amd64.s:330 fp=0xc034d925e8 sp=0xc034d925e0 pc=0x45fe20
runtime.(*mheap).alloc(0x15fbe60, 0x1, 0x110, 0xc03fdffe30)
/home/buildsrv/go1.14beta1/src/runtime/mheap.go:861 +0x81 fp=0xc034d92638 sp=0xc034d925e8 pc=0x424491
runtime.(*mcentral).grow(0x160c6b8, 0x0)
/home/buildsrv/go1.14beta1/src/runtime/mcentral.go:255 +0x79 fp=0xc034d92678 sp=0xc034d92638 pc=0x416d19
runtime.(*mcentral).cacheSpan(0x160c6b8, 0xc034d926f8)
/home/buildsrv/go1.14beta1/src/runtime/mcentral.go:106 +0x2bc fp=0xc034d926c0 sp=0xc034d92678 pc=0x41684c
runtime.(*mcache).refill(0x2ae2647d0, 0x10)
/home/buildsrv/go1.14beta1/src/runtime/mcache.go:138 +0x85 fp=0xc034d926e0 sp=0xc034d926c0 pc=0x416335
runtime.(*mcache).nextFree(0x2ae2647d0, 0xc03df87e10, 0x24, 0x24, 0xc034d927d0)
/home/buildsrv/go1.14beta1/src/runtime/malloc.go:866 +0x87 fp=0xc034d92718 sp=0xc034d926e0 pc=0x40b7f7
runtime.mallocgc(0x70, 0xe24ae0, 0xc034d92801, 0x4f4123)
/home/buildsrv/go1.14beta1/src/runtime/malloc.go:1034 +0x793 fp=0xc034d927b8 sp=0xc034d92718 pc=0x40c133
runtime.newobject(0xe24ae0, 0x24)
/home/buildsrv/go1.14beta1/src/runtime/malloc.go:1163 +0x38 fp=0xc034d927e8 sp=0xc034d927b8 pc=0x40c528
cmd/compile/internal/ssa.(*Func).newValue(0xc03fb0f8c0, 0x87f, 0xc03fdfc5a0, 0xc012551c40, 0x0, 0xc001961bc0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/ssa/func.go:205 +0x224 fp=0xc034d92848 sp=0xc034d927e8 pc=0x63fda4
cmd/compile/internal/ssa.(*Block).NewValue1A(0xc012551c40, 0x0, 0x87f, 0xc03fdfc5a0, 0xe32c40, 0xc014bc2180, 0xc01251b160, 0xc034d92978)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/ssa/func.go:399 +0x5d fp=0xc034d92888 sp=0xc034d92848 pc=0x64127d
cmd/compile/internal/gc.(*state).entryNewValue1A(...)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:860
cmd/compile/internal/gc.(*state).addr(0xc03fb7b320, 0xc001967680, 0xc0000bfe00, 0x0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:4585 +0x2d1 fp=0xc034d92988 sp=0xc034d92888 pc=0xcddc61
cmd/compile/internal/gc.(*state).expr(0xc03fb7b320, 0xc001967680, 0x0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:2007 +0x143 fp=0xc034d92e78 sp=0xc034d92988 pc=0xcc37d3
cmd/compile/internal/gc.(*state).storeArgWithBase(0xc03fb7b320, 0xc001967680, 0xc000447b60, 0xc01251b0f0, 0x18)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:5047 +0xd9 fp=0xc034d92ec8 sp=0xc034d92e78 pc=0xce2329
cmd/compile/internal/gc.(*state).storeArg(0xc03fb7b320, 0xc001967680, 0xc000447b60, 0x18)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:5028 +0x52 fp=0xc034d92f00 sp=0xc034d92ec8 pc=0xce2232
cmd/compile/internal/gc.(*state).call(0xc03fb7b320, 0xc009eb9780, 0x0, 0x4134f7)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:4466 +0x1f1 fp=0xc034d930a0 sp=0xc034d92f00 pc=0xcdb871
cmd/compile/internal/gc.(*state).expr(0xc03fb7b320, 0xc009eb9780, 0x0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:2694 +0x2acd fp=0xc034d93590 sp=0xc034d930a0 pc=0xcc615d
cmd/compile/internal/gc.(*state).stmt(0xc03fb7b320, 0xc009eb9680)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:1239 +0x234 fp=0xc034d937e8 sp=0xc034d93590 pc=0xcbea04
cmd/compile/internal/gc.(*state).stmtList(0xc03fb7b320, 0xc005053740)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:1018 +0x58 fp=0xc034d93820 sp=0xc034d937e8 pc=0xcbe798
cmd/compile/internal/gc.(*state).stmt(0xc03fb7b320, 0xc009eb8180)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:1394 +0x25d2 fp=0xc034d93a78 sp=0xc034d93820 pc=0xcc0da2
cmd/compile/internal/gc.(*state).stmtList(0xc03fb7b320, 0xc0050755e0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:1018 +0x58 fp=0xc034d93ab0 sp=0xc034d93a78 pc=0xcbe798
cmd/compile/internal/gc.(*state).stmt(0xc03fb7b320, 0xc009e82380)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:1450 +0x3d41 fp=0xc034d93d08 sp=0xc034d93ab0 pc=0xcc2511
cmd/compile/internal/gc.(*state).stmtList(0xc03fb7b320, 0xc0050755c0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:1018 +0x58 fp=0xc034d93d40 sp=0xc034d93d08 pc=0xcbe798
cmd/compile/internal/gc.buildssa(0xc009e35340, 0x2, 0x0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:426 +0xc39 fp=0xc034d93ed0 sp=0xc034d93d40 pc=0xcbb1d9
cmd/compile/internal/gc.compileSSA(0xc009e35340, 0x2)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/pgen.go:298 +0x5d fp=0xc034d93f98 sp=0xc034d93ed0 pc=0xc83f2d
cmd/compile/internal/gc.compileFunctions.func2(0xc014b942a0, 0xc008ceb2e0, 0x2)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/pgen.go:363 +0x49 fp=0xc034d93fc8 sp=0xc034d93f98 pc=0xd4b6d9
runtime.goexit()
/home/buildsrv/go1.14beta1/src/runtime/asm_amd64.s:1375 +0x1 fp=0xc034d93fd0 sp=0xc034d93fc8 pc=0x461db1
created by cmd/compile/internal/gc.compileFunctions
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/pgen.go:361 +0x128

goroutine 1 [chan send]:
cmd/compile/internal/gc.compileFunctions()
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/pgen.go:369 +0x17c
cmd/compile/internal/gc.Main(0xe6bbe0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/main.go:724 +0x33a5
main.main()
/home/buildsrv/go1.14beta1/src/cmd/compile/main.go:50 +0xac

goroutine 114 [runnable]:
cmd/compile/internal/ssa.phielimValue(0xc03da9ae00, 0x1)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/ssa/phielim.go:35 +0x126
cmd/compile/internal/ssa.applyRewrite(0xc03c441e40, 0xe6c2a8, 0xe6c320)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/ssa/rewrite.go:38 +0x3fa
cmd/compile/internal/ssa.lower(0xc03c441e40)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/ssa/lower.go:10 +0x46
cmd/compile/internal/ssa.Compile(0xc03c441e40)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/ssa/compile.go:92 +0x9a5
cmd/compile/internal/gc.buildssa(0xc008e46160, 0x3, 0x0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:444 +0xcd8
cmd/compile/internal/gc.compileSSA(0xc008e46160, 0x3)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/pgen.go:298 +0x5d
cmd/compile/internal/gc.compileFunctions.func2(0xc014b942a0, 0xc008ceb2e0, 0x3)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/pgen.go:363 +0x49
created by cmd/compile/internal/gc.compileFunctions
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/pgen.go:361 +0x128

goroutine 80 [runnable]:
sync.runtime_Semrelease(0xc00017c054, 0x1, 0x1)
/home/buildsrv/go1.14beta1/src/runtime/sema.go:66 +0x3e
sync.(*Mutex).unlockSlow(0xc00017c050, 0xc00000000c)
/home/buildsrv/go1.14beta1/src/sync/mutex.go:224 +0xb3
sync.(*Mutex).Unlock(...)
/home/buildsrv/go1.14beta1/src/sync/mutex.go:190
cmd/internal/obj.(*Link).LookupABIInit(0xc00017c000, 0xc03df87da0, 0x24, 0xe40201, 0xc019315170, 0xc0004244b0)
/home/buildsrv/go1.14beta1/src/cmd/internal/obj/sym.go:110 +0xd0
cmd/compile/internal/types.(*Sym).Linksym(0xc009fc6660, 0xc009f8fe00)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/types/sym.go:92 +0x9a
cmd/compile/internal/gc.(*state).call(0xc03fae4900, 0xc009fc8980, 0xc0124e9f00, 0xc03ff22a80)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:4486 +0x2cf
cmd/compile/internal/gc.(*state).expr(0xc03fae4900, 0xc009fc8980, 0x0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:2694 +0x2acd
cmd/compile/internal/gc.(*state).stmt(0xc03fae4900, 0xc00d585100)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:1239 +0x234
cmd/compile/internal/gc.(*state).stmtList(0xc03fae4900, 0xc0050ab1e0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:1018 +0x58
cmd/compile/internal/gc.(*state).stmt(0xc03fae4900, 0xc009fa2580)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:1450 +0x3d41
cmd/compile/internal/gc.(*state).stmtList(0xc03fae4900, 0xc0050ab1a0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:1018 +0x58
cmd/compile/internal/gc.buildssa(0xc009e354a0, 0x1, 0x0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:426 +0xc39
cmd/compile/internal/gc.compileSSA(0xc009e354a0, 0x1)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/pgen.go:298 +0x5d
cmd/compile/internal/gc.compileFunctions.func2(0xc014b942a0, 0xc008ceb2e0, 0x1)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/pgen.go:363 +0x49
created by cmd/compile/internal/gc.compileFunctions
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/pgen.go:361 +0x128

goroutine 79 [runnable]:
cmd/compile/internal/ssa.(*sparseSet).addAll(0xc0279ad980, 0x0, 0x0, 0x0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/ssa/sparseset.go:43 +0x15b
cmd/compile/internal/ssa.(*stackAllocState).computeLive(0xc0150482c0, 0xc03fc38000, 0x896, 0x896)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/ssa/stackalloc.go:316 +0x9df
cmd/compile/internal/ssa.(*stackAllocState).init(0xc0150482c0, 0xc03df8e580, 0xc03fc38000, 0x896, 0x896)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/ssa/stackalloc.go:125 +0x419
cmd/compile/internal/ssa.stackalloc(0xc03df8e580, 0xc03fc38000, 0x896, 0x896, 0x0, 0x0, 0x0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/ssa/stackalloc.go:87 +0xc0
cmd/compile/internal/ssa.(*regAllocState).regalloc(0xc03dfa8f00, 0xc03df8e580)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/ssa/regalloc.go:1660 +0x6f31
cmd/compile/internal/ssa.regalloc(0xc03df8e580)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/ssa/regalloc.go:146 +0x62
cmd/compile/internal/ssa.Compile(0xc03df8e580)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/ssa/compile.go:92 +0x9a5
cmd/compile/internal/gc.buildssa(0xc00b3b9760, 0x0, 0x0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/ssa.go:444 +0xcd8
cmd/compile/internal/gc.compileSSA(0xc00b3b9760, 0x0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/pgen.go:298 +0x5d
cmd/compile/internal/gc.compileFunctions.func2(0xc014b942a0, 0xc008ceb2e0, 0x0)
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/pgen.go:363 +0x49
created by cmd/compile/internal/gc.compileFunctions
/home/buildsrv/go1.14beta1/src/cmd/compile/internal/gc/pgen.go:361 +0x128
go tool dist: FAILED: /home/buildsrv/go1.14beta1/pkg/tool/openbsd_amd64/go_bootstrap install -gcflags=all= -ldflags=all= -a -i cmd/asm cmd/cgo cmd/compile cmd/link: exit status 2

GOGC=70 allows the build to complete successfully (with the fixes mentioned above) in 2GB. I only used 2.5GB earlier to see if the tests would succeed with a little more RAM.

@mknyszek

This comment has been minimized.

Copy link
Contributor

@mknyszek mknyszek commented Dec 20, 2019

TestPageCacheFlush also fails with:
fatal error: failed to reserve page summary memory.

Adding a Skip to TestPageCacheFlush allows the runtime tests to complete with -short.

Good catch. I'll add that one.

Skipping the tests helps, but there is also a noticeable increase in the actual memory requirements for building the compiler under go1.14beta1 with OpenBSD. 2GB memory continues to work for FreeBSD, but not OpenBSD:

$ ./all.bash
Building Go cmd/dist using /home/buildsrv/go1.13.5. (go1.13.5 openbsd/amd64)
Building Go toolchain1 using /home/buildsrv/go1.13.5.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
# cmd/compile/internal/ssa
fatal error: runtime: out of memory

GOGC=70 allows the build to complete successfully (with the fixes mentioned above) in 2GB. I only used 2.5GB earlier to see if the tests would succeed with a little more RAM.

What are you doing to determine if 2 GiB or 2.5 GiB of RAM is enough? Are you trying different machines? Are you varying RLIMIT_DATA?

I suspect this is a combination of the virtual memory use increase (because OpenBSD treats virtual memory like physical memory and limits it per-process) and that the Go compiler might simply be using a little bit more memory than last release. I don't think we're going to take additional steps to curb virtual memory use for this release.

@mpx

This comment has been minimized.

Copy link
Contributor Author

@mpx mpx commented Dec 21, 2019

What are you doing to determine if 2 GiB or 2.5 GiB of RAM is enough? Are you trying different machines? Are you varying RLIMIT_DATA?

I have 2GB RAM VMs. I increased the memory allocation to 2.5GB for testing.

I suspect this is a combination of the virtual memory use increase (because OpenBSD treats virtual memory like physical memory and limits it per-process) and that the Go compiler might simply be using a little bit more memory than last release. I don't think we're going to take additional steps to curb virtual memory use for this release.

To help quantify the differences I've tried compiling 1.13.5 and 1.14beta1 with different data segment limits, decrementing 64MB each time until the build started failing (default GOGC).

  • go1.13.5 builds with 1280MB and fails with 1216MB (cmd/compile/internal/ssa - out of memory)
  • go1.14beta1 builds with 1984MB and fails with 1920MB (cmd/compile/internal/ssa.test - out of memory). This is a 55% increase.

There have been some efforts in the past to slow the growth in requirements so Go can be built on smaller machines. Hopefully there is some way to maintain advantages of the new allocator but reduce requirements on smaller machines? I understand this may not be practical for 1.14.

@gopherbot gopherbot closed this in cd1b9c1 Dec 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.