Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: big map literal creation performance #43020

Open
trams opened this issue Dec 5, 2020 · 1 comment
Open

cmd/compile: big map literal creation performance #43020

trams opened this issue Dec 5, 2020 · 1 comment

Comments

@trams
Copy link

@trams trams commented Dec 5, 2020

I noticed that creating big maps (>=9 elements) using map literals are slow if values are not constant
So this code

return map[string]float {
    "key1": SOME_COMPUTED_ABOVE_VALUE,
    "key2": SOME_COMPUTED_ABOVE_VALUE,
    // more keys here
    "keyN": SOME_COMPUTED_ABOVE_VALUE,
}

works twice as slow then this one

// some code above
result := make(map[string]float, SIZE) // SIZE >= N
result["key1"] = SOME_COMPUTED_ABOVE_VALUE
result["key2"] = SOME_COMPUTED_ABOVE_VALUE
// more keys here
result["keyN"] = SOME_COMPUTED_ABOVE_VALUE
return result

See a post
https://trams.github.io/golang-map-literal-performance/
plus here are benchmarks I used
https://github.com/trams/goplayground/blob/main/map_make_test.go

I asked about this in go developers mailing list and they asked to create a ticket

@josharian mentioned this line in the code cmd/compile/internal/gc/sinit.go:757

What version of Go are you using (go version)?

go1.15.5 linux

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Linux, AMD64

go env Output
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/alex/.cache/go-build"
GOENV="/home/alex/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/alex/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/alex/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/alex/go/src/github.com/trams/goption/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build550590063=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Created a map using map literal

What did you expect to see?

I expected go to allocate enough space for all keys so there won't be any additional allocation and performance would be as good as creating a map manually (and specifying appropriate capacity manually using make)

What did you see instead?

There was not enough space allocated so during creation there was a bunch of additional allocs which is shown by benchmarks

@randall77
Copy link
Contributor

@randall77 randall77 commented Dec 5, 2020

Yes, this is a current limitation of how maps are sized.
We initially size the maps with the number of static entries. An entry is static only when both the key and value are static.
We could really count an entry towards the initial size if just the key is static. The value doesn't have to be.

The organization of this code in the compiler makes it kinda tough, but it should be possible.

@randall77 randall77 added this to the Unplanned milestone Dec 5, 2020
@ALTree ALTree changed the title Big map literal creation performance cmd/compile: big map literal creation performance Dec 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.