runtime: allow FreeBSD users to choose between MADV_FREE and MADV_DONTNEED

[araujobsd]

What version of Go are you using (go version)?

$ go version
go version go1.12.5 freebsd/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="freebsd"
GOOS="freebsd"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build989042882=/tmp/go-build -gno-record-gcc-

What did you do?

On FreeBSD the resident memory of a go binary grows pathologically till the OS rans out of swap and the binary is killed. In some cases, the binary is blocked in uwait state for forever. It takes days to make it happens, but in a machine only running the "go binary" it is possible to simulate this issue with 100% of accuracy.

A way to mitigate this issue is to use MADV_DONTNEED instead of MADV_FREE, after enable it, it is possible to see the resident memory grows and shrinks as much as the "go binary" needs it. Eventually, even when the OS is under memory pressure, I have observed the OS doesn't run out of swap and the "go binary" is not being killed.

The MADV_FREE although has many performance advantages, on FreeBSD it seems to have some downside, specially because it doesn't reduce the resident memory even under OS memory pressure. I think would be useful to provide a GODEBUG option for BSD in the same way it is provided for LINUX, it might help BSD users to mitigate these issues.

Tested with FreeBSD version: HEAD, 12.0-RELEASE-p4, 12-STABLE and 11-STABLE.

[ALTree]

cc @aclements

[gopherbot]

Change https://golang.org/cl/181498 mentions this issue: runtime: Allow users of FreeBSD OS to choose between MADV_FREE and MADV_DONTNEED.