cmd/compile: go statement desugaring results in additional allocations

[qmuntal]

What version of Go are you using (go version)?

go version devel go1.17-cca23a7 Sat May 22 00:51:17 2021 +0000 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\***\AppData\Local\go-build
set GOENV=C:\Users\***\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\***\go\pkg\mod
set GONOPROXY=*.hp.com
set GONOSUMDB=*.hp.com
set GOOS=windows
set GOPATH=C:\Users\***\go
set GOPRIVATE=*.hp.com
set GOPROXY=direct
set GOROOT=C:\Users\***\sdk\gotip
set GOSUMDB=off
set GOTMPDIR=
set GOTOOLDIR=C:\Users\***\sdk\gotip\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=devel go1.17-cca23a7 Sat May 22 00:51:17 2021 +0000
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\Users\***\Documents\code\go3mf-thumbnailer\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\***\AppData\Local\Temp\go-build497339627=/tmp/go-build -gno-record-gcc-switches

What did you do?

package main

import (
  "sync"
  "testing"
)

func BenchmarkAlloc(b *testing.B) {
  count := 10
  for n := 0; n < b.N; n++ {
    var wg sync.WaitGroup
    wg.Add(count)
    for i := 0; i < count; i++ {
        go func() {
          wg.Done()
        }()
     }
     wg.Wait()
  }
}

gotip test -run AAAA -bench BenchmarkAlloc -benchmem

What did you expect to see?

Same result as with go1.16.3 and gotip+GOEXPERIMENT=noregabi:

goos: windows
goarch: amd64
pkg: main
cpu: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
BenchmarkCreate-8         211135              6922 ns/op              16 B/op          1 allocs/op
PASS
ok      main      2.048s

What did you see instead?

Alloc count grow linearly with the size of the loop:

goos: windows
goarch: amd64
pkg: main      
cpu: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
BenchmarkCreate-8         183998              7023 ns/op             176 B/op         11 allocs/op
PASS
ok      main            2.575s

Notice that the number of allocations is directly proportional to the number of iterations inside the loop.

Context

I've found this regression while trying gotip in a private codebase. The provided benchmark is just a synthetic and minimal example I've found that reproduces what I'm seeing in the real code.

This is a memory profile snap for this benchmark, generated using go tool pprof -alloc_objects mem.out:

         .          .      6:)
         .          .      7:
         .          .      8:func BenchmarkCreate(b *testing.B) {
         .          .      9:   count := 10
         .          .     10:   for n := 0; n < b.N; n++ {
    131074     131074     11:           var wg sync.WaitGroup
         .          .     12:           wg.Add(count)
         .          .     13:           for i := 0; i < count; i++ {
   1507351    1507351     14:                   go func() {
         .          .     15:                           wg.Done()
         .          .     16:                   }()
         .          .     17:           }
         .          .     18:           wg.Wait()

[mknyszek]

Thanks for the report.

I believe this is due to a known change in Go 1.17 related to go and defer 'desugaring,' that may result in more allocations in some cases. You mention that you saw this in a larger codebase, but could you tell us more about what you saw there? Specifically, is there a significant performance regression in that case?

CC @thanm @cherrymui

[mknyszek]

Not marking this a release blocker until we have more information. Also because to some degree this is expected, and for the most part we've found the benefits to outweigh the costs overall by a significant margin.

[qmuntal]

I believe this is due to a known change in Go 1.17 related to go and defer 'desugaring,' that may result in more allocations in some cases. You mention that you saw this in a larger codebase, but could you tell us more about what you saw there? Specifically, is there a significant performance regression in that case?

I'm seeing a huge increase of allocations in all multiple parts of the codebase that are called from within a loop an unbounded number of times. In some realistic non-worth-case benchmarks I'm seeing 1000% allocation increase.

The codebase that I'm benchmarking parses 3D software-generated files, such as point clouds or 3D meshes, which can have millions of 3D points. Until now the parsing algorithm had O(1) complexity with respect to the number of vectors, but in gotip is O(N).

On the other hand, I'm also seeing a 12% speed performance gain in most of the benchmarks, which is much appreciated when processing big files. Still, I'm not sure that this speed gain is realistic as there are more allocations so I expect the GC to eat part of this improvement.

[mknyszek]

@qmuntal I think we need more details as to the nature of these heap allocations. Do they all involve a go statement? Or do they look like something else (like defer statements)?

You can use the go build flag -gcflags=-m=2 to get inlining and escape analysis data at build time to help your understand what the new sources of allocation are.

[qmuntal]

@mknyszek Not all the offending lines use go or defer. Here are some examples

val, err := strconv.ParseUint(string(a.Value), 10, 24) // a.Value is a byte slice

if d.OnStart != nil {
   // t is a complex struct holding a byte slice reference
   d.OnStart(t) // <- Ofending!
}

wn := runtime.GOMAXPROCS(0)
var wg sync.WaitGroup
wg.Add(wn)
for wi := 0; wi < wn; wi++ {
  go func(wi int) { // <- Ofending!
    // ...
  }
}

// This comes from https://tip.golang.org/src/mime/mediatype.go?s=3622:3707#152
 params = make(map[string]string)

I will do the inlining and scape analysis tomorrow morning (CET), I'm running out of time now.

Thanks for jumping into this.

[mknyszek]

CC @mdempsky @dr2chase for escape analysis and even more compiler regabi expertise.

[cherrymui]

Yes, with register ABI there are more go'd closures needs allocation now. But I'm not sure whether it is really a problem for performance. We already needs to allocate the new goroutine's stack (which is not counted as memory allocation in profile, but it's there). Now we need to do an extra allocation for that closure (which is typically a much smaller allocation). As you posted, the overall performance is improved.

We could do something clever, e.g. allocating the closure on the new goroutine's stack. But there is a complexity for that. I'm not sure it's worth doing. We could consider that if it is a real problem.

[cherrymui]

The escape analysis isn't really relevant here. For go statement, it runs asynchronously and so in general we always need to allocate that closure. Except the case that it is not a real closure, i.e. it doesn't capture anything. With register ABI, there are more closures become "real closures", in that we need to capture arguments as well.

[mknyszek]

Right, it's not really relevant to the desugaring, I looped in escape analysis people because reacting to @qmuntal claim that lots of other non-go, non-defer sites now end up allocating. But, it's still unclear to me where the new allocations are coming from.

[qmuntal]

My codebase is memory intensive and I've always tried to reduce the allocations in order to get performance gains indirectly. In this case, I haven't seen any performance regression but just much more allocations everywhere, so If you say this is the expected behavior when migrating to regabi then it's ok for me. If in the future I see there is a real performance issue here I can report it back with more data.

I let you decide what to do with this issue, I can either close it or provide you more data to better understand where to new allocations are coming from.

[mknyszek]

In that case, I'll at least move this issue out of the Go 1.17 milestone.