Description
What version of Go are you using (go version
)?
$ go version go version go1.12.7 darwin/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env
)?
go env
Output
$ go env GOARCH="amd64" GOBIN="" GOCACHE="/Users/danielchatfield/Library/Caches/go-build" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="darwin" GOOS="darwin" GOPATH="/Users/danielchatfield" GOPROXY="" GORACE="" GOROOT="/usr/local/Cellar/go/1.12.7/libexec" GOTMPDIR="" GOTOOLDIR="/usr/local/Cellar/go/1.12.7/libexec/pkg/tool/darwin_amd64" GCCGO="gccgo" CC="clang" CXX="clang++" CGO_ENABLED="1" GOMOD="" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/fq/s8x0wv5527g1yz9lltnqzds80000gn/T/go-build127136404=/tmp/go-build -gno-record-gcc-switches -fno-common"
What did you do?
I've written a benchmark that illustrates this performance issue here: https://github.com/danielchatfield/jsonbench
The issue boils down to unmarshaling a JSON object into a struct is O(n^2)
with respect to the number of struct fields. We recently changed some code that previously used a map to be using an autogenerated struct with a large number of fields and observed a very severe performance regression.
On my machine this takes 1.3s of CPU time to unmarshal 10,000 JSON fields.
What did you expect to see?
I expected it to use a reasonable number of CPU cycles.
What did you see instead?
It used an unreasonable number of CPU cycles.
Proposed fix
I have a patch locally that returns this to being O(n)
at the expense of allocating an additional map. I can add a heuristic such that the map is only allocated if the struct has 50 or more fields (where the O(n^2)
complexity starts to bite). Does this seem directionally reasonable? If so, I'll prepare the change and submit it.