encoding/json: map[string]interface performance

[pavolloffay]

What version of Go are you using (go version)?

go version go1.13.4 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ploffay/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/ploffay/projects/golang"
GOPROXY=""
GORACE=""
GOROOT="/home/ploffay/bin/go"
GOTMPDIR=""
GOTOOLDIR="/home/ploffay/bin/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build014168030=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Benchmark JSON marshaling and unmarshaling for encoding/json, gojay, jsoninter, fastjson. The performance of stdlib is overall similar to other 3rd party libraries. However, marshaling produces substantially more allocations than gojay library for map[string]interface{} and two times ns/op.

For unmarshaling the allocations by gojay are 30% lower (my number not amount) and ns/op half time of stdlib.

marshaling:

GOMAXPROCS=1 go test -bench=Marshal -test.benchtime=3s -benchmem -cpuprofile profile_cpu.out ./pkg/jsontest
goos: linux
goarch: amd64
pkg: json-benchmark/pkg/jsontest
BenchmarkMarshalStdlib/default.json        	  558175	      5728 ns/op	    1536 B/op	       1 allocs/op
BenchmarkMarshalStdlib/default-unicode.json         	  768222	      4183 ns/op	    1024 B/op	       1 allocs/op
BenchmarkMarshalStdlib/default-tagmap.json          	  362587	      8962 ns/op	    2752 B/op	      37 allocs/op

BenchmarkMarshalGojay/default.json         	  516730	      6654 ns/op	    4240 B/op	       9 allocs/op
BenchmarkMarshalGojay/default-unicode.json 	  904315	      3617 ns/op	    1808 B/op	       8 allocs/op
BenchmarkMarshalGojay/default-tagmap.json  	  802406	      4899 ns/op	    2607 B/op	       8 allocs/op

BenchmarkMarshalJsoniter/default.json               	  499992	      6932 ns/op	    1800 B/op	       6 allocs/op
BenchmarkMarshalJsoniter/default-unicode.json       	  612794	      5121 ns/op	    1288 B/op	       6 allocs/op
BenchmarkMarshalJsoniter/default-tagmap.json        	  245335	     13910 ns/op	    7930 B/op	      50 allocs/op

unmarshaling

GOMAXPROCS=4 go test -bench=Unmarshal -test.benchtime=3s -benchmem -cpuprofile profile_cpu.out ./pkg/jsontest
goos: linux
goarch: amd64
pkg: json-benchmark/pkg/jsontest
BenchmarkUnmarshalStdlib/default.json-4           	  123186	     29579 ns/op	    1520 B/op	      73 allocs/op
BenchmarkUnmarshalStdlib/default-unicode.json-4   	  158065	     21486 ns/op	    1136 B/op	      53 allocs/op
BenchmarkUnmarshalStdlib/default-tagmap.json-4    	  146869	     21530 ns/op	    2192 B/op	      95 allocs/op

BenchmarkUnmarshalGojay/default.json-4            	  194344	     18322 ns/op	    9983 B/op	      75 allocs/op
BenchmarkUnmarshalGojay/default-unicode.json-4    	  285207	     11772 ns/op	    6932 B/op	      49 allocs/op
BenchmarkUnmarshalGojay/default-tagmap.json-4     	  287793	     12263 ns/op	    5470 B/op	      61 allocs/op

BenchmarkUnmarshalJsoninter/default.json-4        	  379986	      8116 ns/op	    1184 B/op	      76 allocs/op
BenchmarkUnmarshalJsoninter/default-unicode.json-4         	  571060	      5947 ns/op	     816 B/op	      56 allocs/op
BenchmarkUnmarshalJsoninter/default-tagmap.json-4          	  523004	      6824 ns/op	    1456 B/op	      85 allocs/op

BenchmarkUnmarshalFastjson/default.json-4         	  161304	     20712 ns/op	   29440 B/op	     158 allocs/op
BenchmarkUnmarshalFastjson/default-unicode.json-4 	  251964	     13733 ns/op	   16160 B/op	     115 allocs/op
BenchmarkUnmarshalFastjson/default-tagmap.json-4  	  332452	      9770 ns/op	   14416 B/op	      54 allocs/op

Benchmarks and results are here https://github.com/pavolloffay/golang-json-benchmark

What did you expect to see?

encoding/json the same performance as https://github.com/francoispqt/gojay

What did you see instead?

encoding/json is slower than https://github.com/francoispqt/gojay for marshaling map[string]interface{} and for unmarshaling it produces more allocations and is 2 times slower on ns/op.

[mvdan]

Why is the encoding/decoding performance of maps important here? Usually, when one is decoding large amounts of data, structs are used. encoding/json is already pretty efficient at encoding and decoding those.

It seems to me like gojay makes types like maps more efficient by adding more code to handle them especially. It's something we can consider, but it's a tradeoff. We don't want the standard library to get large in size, and hard to maintain.

Do you have a realistic piece of code where the peformance here is a problem, or is this simply about benchmarking different json libraries?

[pavolloffay]

We use maps to store data in our data model. The map is used to allow queries in Kibana - kibana does not support query for nested objet array.

[mvdan]

I see, thanks. If someone wants to experiment with this, I'll be happy to review the code. There might be low hanging fruit to remove an alloc or two in some cases.

[rsc]

It looks like gojay avoids allocating by unsafely converting the []byte input to a string.
https://github.com/francoispqt/gojay/blob/1398296d938f9fae26750ddc2fe356b6d897f799/decode_object.go#L259
This means that the caller cannot modify or reuse the []byte containing the input JSON until all references to those strings have gone away, so basically never. That is, a client who passes a []byte to gojay must never change that data again, without very careful tracking of where the results of the unmarshal have gotten to. (And this is undocumented.)

This is not a condition encoding/json imposes on its callers, nor can it start. So this particular "optimization" ("bug" might be more likely) is not available.

Gojay also seems to be accidentally quadratic in its unescaping of strings, for what it's worth.

I'm not sure there's anything actionable in this issue.

[andybons]

Closing as this doesn’t seem actionable.