This performance bug can be seen in the latest Golang today on specific (common) current hardware.
Currently, Golang's global free list is not segregated into NUMA nodes. Work stealing also not NUMA-node aware.
A 2x penalty in memory accesses can be seen on NUMA-node misses (ref). Today, AMD EPYC exposes 4 NUMA nodes. Modern chiplet economics suggest NUMA is a growing trend.
Go could either (or some mix of):
- Do nothing, forcing devs into the penalty (as I'm seeing today on a specialty cloud deployment),
- Recommend CGroups to lock a process to CPUs or NUMA nodes (sounds awful to use and not cross-platform),
- Segregate operations & work-stealing completely (could starve unnecessarily & run-up free-list size),
- Prefer its NUMA node, but fallback to using others for free-list & work-stealing,
- Use some more clever "cross-NUMA sometimes" logic, or
- Expose it to the developer somehow: today's
LockOSThread() doesn't apply to new().
Allocations in the free. list can come from anywhere. Even if new() was NUMA-specific, once a GoRoutine is stolen across NUMA zones, not-only will most of its previous RAM accesses get 2x slower, but new allocations give it no particular "home NUMA" node, which can create a piece of hot code that is working between 2 different NUMA nodes. If GoRoutines have a "home P" then memory could be allocated there and it could be returned there once stolen & paused in the hopes that it will generally stay in that node.
Ultimately, Go cannot protect developers from the experience of this bottleneck as I can trivially create 2 big lists on different 'P' and diff them in a way that gets me crossing the NUMA boundaries. Exposing this to developers could be as simple as presuming most new() allocations are "in-node" and then allocating memory before sending it to whoever would fill it (getting penalized), then returning processing to who will use it intensively. This could create a culture of "allocate what you plan to use intensively".
This performance bug can be seen in the latest Golang today on specific (common) current hardware.
Currently, Golang's global free list is not segregated into NUMA nodes. Work stealing also not NUMA-node aware.
A 2x penalty in memory accesses can be seen on NUMA-node misses (ref). Today, AMD EPYC exposes 4 NUMA nodes. Modern chiplet economics suggest NUMA is a growing trend.
Go could either (or some mix of):
LockOSThread()doesn't apply tonew().Allocations in the free. list can come from anywhere. Even if new() was NUMA-specific, once a GoRoutine is stolen across NUMA zones, not-only will most of its previous RAM accesses get 2x slower, but new allocations give it no particular "home NUMA" node, which can create a piece of hot code that is working between 2 different NUMA nodes. If GoRoutines have a "home P" then memory could be allocated there and it could be returned there once stolen & paused in the hopes that it will generally stay in that node.
Ultimately, Go cannot protect developers from the experience of this bottleneck as I can trivially create 2 big lists on different 'P' and diff them in a way that gets me crossing the NUMA boundaries. Exposing this to developers could be as simple as presuming most new() allocations are "in-node" and then allocating memory before sending it to whoever would fill it (getting penalized), then returning processing to who will use it intensively. This could create a culture of "allocate what you plan to use intensively".