Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: allow FreeBSD users to choose between MADV_FREE and MADV_DONTNEED #32519

Open
araujobsd opened this issue Jun 10, 2019 · 2 comments · May be fixed by #32520

Comments

@araujobsd
Copy link

commented Jun 10, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.5 freebsd/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="freebsd"
GOOS="freebsd"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build989042882=/tmp/go-build -gno-record-gcc-

What did you do?

On FreeBSD the resident memory of a go binary grows pathologically till the OS rans out of swap and the binary is killed. In some cases, the binary is blocked in uwait state for forever. It takes days to make it happens, but in a machine only running the "go binary" it is possible to simulate this issue with 100% of accuracy.

A way to mitigate this issue is to use MADV_DONTNEED instead of MADV_FREE, after enable it, it is possible to see the resident memory grows and shrinks as much as the "go binary" needs it. Eventually, even when the OS is under memory pressure, I have observed the OS doesn't run out of swap and the "go binary" is not being killed.

The MADV_FREE although has many performance advantages, on FreeBSD it seems to have some downside, specially because it doesn't reduce the resident memory even under OS memory pressure. I think would be useful to provide a GODEBUG option for BSD in the same way it is provided for LINUX, it might help BSD users to mitigate these issues.

Tested with FreeBSD version: HEAD, 12.0-RELEASE-p4, 12-STABLE and 11-STABLE.

@ALTree

This comment has been minimized.

Copy link
Member

commented Jun 10, 2019

@ALTree ALTree changed the title runtime: Allow users of FreeBSD OS to choose between MADV_FREE and MADV_DONTNEED. runtime: allow FreeBSD users to choose between MADV_FREE and MADV_DONTNEED Jun 10, 2019
@gopherbot

This comment has been minimized.

Copy link

commented Jun 10, 2019

Change https://golang.org/cl/181498 mentions this issue: runtime: Allow users of FreeBSD OS to choose between MADV_FREE and MADV_DONTNEED.

@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.