runtime: false-sharing observed on mheap for high core count environment

[gavinlichn]

What version of Go are you using (go version)?

$ go version
go version devel go1.22-ac64a3628b Sat Aug 12 03:56:58 2023 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/root/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/root/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/home/ansible/ligang/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/home/ansible/ligang/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='devel go1.22-ac64a3628b Sat Aug 12 03:56:58 2023 +0000'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/ansible/ligang/minio/minio_svr/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3702239426=/tmp/go-build -gno-record-gcc-switches'

What did you do?

I benchmarked MinIO with master version of go runtime.
Server: https://github.com/minio/minio
Client: https://github.com/minio/warp
Configuration:

• Single node MinIO Server, without TLS enabled,
• Run warp GET request, 128KB object and 512 concurrent
• GOGC=1000
• 2 socket server(56c per socket),
• pin MinIO Server and warp with specific CPU cores
• MinIO bind with 6 tmpfs ramdisks

What did you expect to see?

No significant false-sharing

What did you see instead?

While pin MinIO server with high core count, significant false-sharing observed.
While pin MinIO with 100 CPU cores(0-99), Pin warp with 12 CPU Core(100-111) perf c2c result as below(omited some minor info)

#        ----- HITM -----  -- Store Refs --  -- Data address -- ---------- cycles ----------    Total       cpu                                           Shared               
#   Num  RmtHitm  LclHitm   L1 Hit  L1 Miss       Offset  Node  rmt hitm  lcl hitm      load  records       cnt                                   Symbol  Object  Source:Line  
# .....  .......  .......  .......  .......  ...........  ....  ........  ........  ........  .......  ........  .......................................  .....  ...........
  ------------------------------------------------------
      1      398     1056       31        0    0x62b2640
  ------------------------------------------------------
           2.26%    0.57%    3.23%    0.00%          0x0     0      4582      3858      3165      215        42  [.] runtime.(*mheap).initSpan            minio  types.go:420  
           2.26%    1.14%   41.94%    0.00%          0x8     0      4356      2245      3288      330        42  [.] runtime.(*sweepLocked).sweep         minio  types.go:344  
           1.76%    0.09%   16.13%    0.00%         0x30     0      4728      1534      3506      180        41  [.] runtime.(*mheap).reclaim             minio  types.go:401  
          63.07%   63.92%    0.00%    0.00%         0x38     0      1859      1646      1931     5996        42  [.] runtime.writeHeapBits.flush          minio  mbitmap.go:865
           6.03%    8.05%    0.00%    0.00%         0x38     0      2464      3614      2256     6443        42  [.] runtime.writeHeapBits.flush          minio  mbitmap.go:893
           5.28%    6.63%    0.00%    0.00%         0x38     0      2050      1462      2166      812        42  [.] runtime.writeHeapBits.write          minio  mbitmap.go:812
           1.01%    2.37%    0.00%    0.00%         0x38     0      2540      1707      2348      222        42  [.] runtime.spanOfHeap                   minio  mheap.go:700  

[gopherbot]

Change https://go.dev/cl/525955 mentions this issue: runtime: Resolve false sharing for frequent memory allocate workloads