You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
During an investigation into a performance issue with our site, we narrowed down the issue to our JSON parsing code taking an extremely long time to parse larger JSON files (multiple minutes to parse a 30MB file). Running a series of benchmarks seems to show that using json.NewDecoder(...).decode(...) is significantly slower than reading the contents into memory and then using json.Unmarshal(...). In fact it seems significantly slower than even the standard library's json.NewDecoder(...).decode(...).
The following are the results of benchmarking the (Unmarshal | Decode) methods of the (encoding/json | goccy/go-json) libraries with a typical file we use:
$ go test -bench=.
goos: darwin
goarch: arm64
pkg: bench
BenchmarkUnmarshalEncodingJson-10 278 4217963 ns/op
BenchmarkUnmarshalGoJson-10 723 1619416 ns/op
BenchmarkDecodeEncodingJson-10 292 4070073 ns/op
BenchmarkDecodeGoJson-10 2 531405062 ns/op
PASS
ok bench 6.261s
During an investigation into a performance issue with our site, we narrowed down the issue to our JSON parsing code taking an extremely long time to parse larger JSON files (multiple minutes to parse a 30MB file). Running a series of benchmarks seems to show that using
json.NewDecoder(...).decode(...)
is significantly slower than reading the contents into memory and then usingjson.Unmarshal(...)
. In fact it seems significantly slower than even the standard library'sjson.NewDecoder(...).decode(...)
.The following are the results of benchmarking the (
Unmarshal | Decode
) methods of the (encoding/json | goccy/go-json
) libraries with a typical file we use:I've attached the benchmarking code:
I've also attached the test JSON file: test.json
Are there any configurations or settings we should set on the Decoder to fix this performance issue?
The text was updated successfully, but these errors were encountered: