cmd/compile: big map literal creation performance

[trams]

I noticed that creating big maps (>=9 elements) using map literals are slow if values are not constant
So this code

return map[string]float {
    "key1": SOME_COMPUTED_ABOVE_VALUE,
    "key2": SOME_COMPUTED_ABOVE_VALUE,
    // more keys here
    "keyN": SOME_COMPUTED_ABOVE_VALUE,
}

works twice as slow then this one

// some code above
result := make(map[string]float, SIZE) // SIZE >= N
result["key1"] = SOME_COMPUTED_ABOVE_VALUE
result["key2"] = SOME_COMPUTED_ABOVE_VALUE
// more keys here
result["keyN"] = SOME_COMPUTED_ABOVE_VALUE
return result

See a post
https://trams.github.io/golang-map-literal-performance/
plus here are benchmarks I used
https://github.com/trams/goplayground/blob/main/map_make_test.go

I asked about this in go developers mailing list and they asked to create a ticket

@josharian mentioned this line in the code cmd/compile/internal/gc/sinit.go:757

What version of Go are you using (go version)?

go1.15.5 linux

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Linux, AMD64

go env Output

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/alex/.cache/go-build"
GOENV="/home/alex/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/alex/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/alex/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/alex/go/src/github.com/trams/goption/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build550590063=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Created a map using map literal

What did you expect to see?

I expected go to allocate enough space for all keys so there won't be any additional allocation and performance would be as good as creating a map manually (and specifying appropriate capacity manually using make)

What did you see instead?

There was not enough space allocated so during creation there was a bunch of additional allocs which is shown by benchmarks

[randall77]

Yes, this is a current limitation of how maps are sized.
We initially size the maps with the number of static entries. An entry is static only when both the key and value are static.
We could really count an entry towards the initial size if just the key is static. The value doesn't have to be.

The organization of this code in the compiler makes it kinda tough, but it should be possible.

[trams]

I just revisited this with a newer version of go compiler and I still observe this effect with 1.17.6
I thought I would just share this here

[gopherbot]

Change https://golang.org/cl/383634 mentions this issue: cmd/compile: include all entries in map literal hint size