runtime: set runtime thread names

[CAFxX]

Whenever possible it would be ideal to have the go runtime set thread names to its threads, so that it becomes easier for newcomers to understand what the different threads are.

In the screenshot below, taken on a small machine with 4 cores, it is unclear at a glance why the go process has 9 threads started (no GOMAXPROCS is set, so I have seen newcomers expect it would start at most 4). If the runtime opportunistically set the names of the threads (even just e.g. "runtime", "application", "blocked-syscall", etc.) it would help not just avoid misconceptions, but it may also help speed up troubleshooting in certain scenarios.

This can understandably be filed under "nice-to-haves", but it's something I have definitely ran into multiple times throughout the years, so I'm throwing it out there for consideration. To be clear, I am not suggesting to allow users to set custom thread names: the runtime would be in control of the names.

[ianlancetaylor]

Are you suggesting that the Go runtime regularly change the name of each thread to reflect its current state? What mechanism would we use to do that?

(We do have other mechanisms like the runtime/trace package that provide this kind of information in what I hope would be a more useful way.)

[CAFxX]

Are you suggesting that the Go runtime regularly change the name of each thread to reflect its current state? What mechanism would we use to do that?

I would expect some issues if targeting "absolute precision", so I would shoot for something still useful even if not as precise (I will be handwaving a bit here, because I'm not familiar with all corners of the runtime), e.g.:

• For application code that briefly enters the runtime (e.g. for memory allocations, goroutine scheduling, gc assists, etc.) no. Those threads would be just marked something generic like "application" even while they're transiently in the runtime.
• For runtime threads that have a single purpose for their lifetime, give them a descriptive name and that's it.
• For threads stuck in a syscall, opportunistically mark/unmark them as stuck when we realize they are stuck/unstuck
• For threads that change role "infrequently", mark them when they change role
• For threads that change role too frequently to accept the prctl syscall overhead, either approximate their role to the most common one and avoid tracking changes, or come up with an ad-hoc mechanism

(note: I'm not claiming all the above cases exist today; it is quite possible only a couple of those cases exist, I haven't followed runtime work closely in the last ~2 years)

(We do have other mechanisms like the runtime/trace package that provide this kind of information in what I hope would be a more useful way.)

And these are very useful when policies allow their use, but this isn't always the case. Furthermore, at the very least in my experience, as a go user there is value in being able to quickly distinguish between, e.g.:

• "a single one of my application threads is running at 100% cpu utilization"
• "all my application threads are almost idle"
• "I have many threads stuck in syscalls"
• etc.

and, in general, being able to answer the common newcomer questions: why do I have N > GOMAXPROCS threads in my process? what are these other threads for? what are they doing?

[MikeMitchellWebDev]

There can be up to 10,000 threads (see sched.maxmcount in proc.go). Might be impractical to name them. Using a debugger like delve with a threads command can help.

[ianlancetaylor]

Calling the prctl system call every time a thread moves in and out of the scheduler is going to be too expensive for a feature that doesn't matter to 99.9% of Go users.

[CAFxX]

I think this can be mitigated, e.g. set the name of a thread only if it needs to be changed, change the name only up to a few times per seconds (for example, from a dedicated grouting that only executes the update once every 250ms). I'm sure we can work something out that will have negligible impact.

As for the suggestion of using a debugger, see my point above about policies preventing that.

[ianlancetaylor]

I'm not yet seeing the advantage of an inaccurate thread name. Who does that help?

[CAFxX]

There's a few different reason why I think that would be an acceptable tradeoff:

• any tool that users use will not refresh in real-time; top updates e.g. every second or so even if there's a slight delay in the update time that causes the name of the thread to be temporarily wrong it won't matter in practice
• related to the previous point, if a thread keeps switching between roles, users won't be able to see it in real time anyway, but just the information that the thread is often switching (because e.g. every time top refreshes the name changes) would be a useful signal when troubleshooting.

[ianlancetaylor]

Remember that goroutines are multiplexed onto threads. In practice every thread will be switching constantly. It would be downright unusual for a thread to not be switching. About the only way that could happen would be a call into C code that takes a long time to run.

[mknyszek]

In triage, we agree with @ianlancetaylor; this doesn't seem like something that's worth the resource costs. If we let the names get inaccurate or go stale I would imagine that only creates more confusion.

To be clear, the runtime doesn't really have dedicated threads for things. Goroutines are constantly swapping between threads; this includes things like garbage collection workers, which are goroutines, not threads. There is maybe one thread we could name (sysmon) but that's not very useful to anybody except those debugging the runtime, and we have other ways of finding sysmon.

[mknyszek]

Given that there doesn't seem to be anything actionable here, I'm going to close the issue. Please feel free to comment if there's something we're misunderstanding or some new information that might make such a feature more viable. Thanks.