proposal: runtime: allow access to stopTheWorld/startTheWorld via go:linkname

[fjl]

Proposal Details

My module github.com/fjl/memsize walks a data structure and computes its total referred memory size in bytes, as well as producing a report of per-type memory usage. This is meant to be used for debugging memory leaks in live programs in a similar manner to taking a profile over HTTP with pprof, and to my knowledge there exists no other comparable tool.

memsize requires access to runtime.stopTheWorld and runtime.startTheWorld in order to safely walk the object graph without racing with the program that is being debugged. Up until Go version 1.22, I was able to call these functions via go:linkname. Go 1.23 disallows this, and as per #67401 only specifically approved entry points can be linkname'd.

Please allow access to runtime.stopTheWorld and runtime.startTheWorld by adding a go:linkname directive for Go 1.23.

Alternatively, we could consider adding an exposed API, perhaps in package runtime/debug, to perform a similar task. I could envision something like:

func WithWorldStopped(reason string, f func())

Perhaps access to stopTheWorld/startTheWorld could be granted temporarily during the Go 1.23 cycle while we work out a replacement API.

xref fjl/memsize#4
xref #67401

[fjl]

BTW, I wasn't sure which issue category was appropriate here. Originally went with Bug, but it didn't really fit either. Proposal seems kind of strongly-worded, I really just want to get my request in.

[seankhliao]

given it is for debugging use, I didn't see an issue with requiring the checklinkname flag during a build.

[fjl]

memsize is meant to be integrated into production executables, i.e. you typically set up the memsize web interface handler on the pprof endpoint and then access it the same way you'd access that. It's not like you'd specificallly build a debugging executable to use pprof. Most people just have it always enabled so they can go in and debug their service when it is in a state of failure.

[ianlancetaylor]

CC @golang/runtime

[navytux]

For the reference: go123's xruntime also hit this regression when trying to add support for go1.23: there stopTheWorld is used to serialize modification of program tracepoints wrt the program running itself:

https://lab.nexedi.com/kirr/go123/-/blob/8299741f/tracing/internal/xruntime/runtime.go#L24-38
https://lab.nexedi.com/kirr/go123/-/blob/8299741f/tracing/tracing.go#L211-225
https://pkg.go.dev/lab.nexedi.com/kirr/go123@v0.0.0-20240626173136-48920809d24c/tracing

lab.nexedi.com/kirr/go123/tracing was working ok since 2017 and it would be good to preserve support for that.

Thanks beforehand,
Kirill

[rsc]

stopTheWorld is very sensitive and difficult to use correctly.
We are very unlikely to support calling it even using linkname.
This is the kind of dangerous, difficult-to-keep-working use that
the new linkname restrictions are meant to catch.

[aktau]

Would an enhanced viewcore be an alternative? The desire to improve this is mentioned in #43930 (comment) (cc @mknyszek). If I'm not mistaken, the tracking issue is either #57447 or #45631.

Enhanced viewecore (gocore) would not support live debugging, but in my experience this is often just a nice-to-have. Most deployments have multiple concurrent tasks, managed by a cluster scheduler (Kubernetes, ...). Tasks go up and down for various reasons (machine reboot, crash, lack of resources, ...) and the job in its entirety should be tolerant to tasks disappearing. Hence, sending a signal to crash and produce a core dump (or heap dump) would be a decent way to identify the exact nature of leaks, for us.

Until this type of functionality is available, we're relegated to reading the code from the allocation stack of the leaked object (likely identified by diffing two heap profiles). This is (far) less productive for identifying leaks than the proposal or an improved core viewer.

[fjl]

Enhanced viewcore would be nice, but it's kind of orthogonal and doesn't cover some use cases. I often use memsize by to take snapshot reports over time, checking for trends in the number of reachable objects. Can't do that if the program has to crash to get the heap.

[navytux]

@rsc, thanks for feedback.

While I completely understand the desire of Go team to close access to private
functionality, my reading of #67401 is that
the access will not be closed to what is already used from outside to prevent
breaking existing applications and libraries. In other words for practical
reason the set of symbols, that is already used from outside, will continue to
be provided via go:linkname, and it is only new symbols, and symbols that were
not used previously, that become inaccessible via go:linkname to prevent
growing the set of private API that is exported in reality.

I agree that in ideal world the set of private API that is exported in reality
should be empty, and that reaching that empty set is the long-term goal.

However here we are not starting from scratch. Over the years people used to
have go:linkname working and started to depend on functionality, for which
there is usually no public equivalent. For example in my tracing library with
Go1.23 dropping access to stopTheWorld I'll have a hard time to find out what
to do.

So I suggest to please reconsider providing access to stopTheWorld via go:linkname.

It has been only a few days since go1.23rc1 and it is already two projects
speaking here due to breakage. I'm sure there will be more over time.

Kirill

[rsc]

@fjl I don't fully understand why pprof isn't enough to catch memory leaks. Usually when there's a memory leak it shows up as the dominant memory source in a pprof heap profile.