New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimize dynamic unmarshaling #1327
Conversation
Signed-off-by: Eric Chlebek <eric@sensu.io>
Signed-off-by: Eric Chlebek <eric@sensu.io>
Signed-off-by: Eric Chlebek <eric@sensu.io>
Signed-off-by: Eric Chlebek <eric@sensu.io>
Signed-off-by: Eric Chlebek <eric@sensu.io>
* Use sync.Pool in several places to optimize allocations. * Use a wrapper type and json.RawMessage to reduce work needed in unmarshaling dynamic attributes. * Use a slice instead of a map for getJSONFields. This commit substantially reduces runtime and allocations for dynamic json unmarshaling. Before: [15:34 eric ~/.../sensu-go/types ((6cfa8e40...) $%)]$ go test -bench=. -benchmem goos: linux goarch: amd64 pkg: github.com/sensu/sensu-go/types BenchmarkCheckRequestMarshal-4 300000 3915 ns/op 3624 B/op 30 allocs/op BenchmarkCheckRequestUnmarshal-4 30000 44284 ns/op 39002 B/op 386 allocs/op BenchmarkCheckConfigMarshal-4 500000 2399 ns/op 2056 B/op 19 allocs/op BenchmarkCheckConfigUnmarshal-4 30000 40347 ns/op 37340 B/op 358 allocs/op PASS ok github.com/sensu/sensu-go/types 5.901s After: [15:35 eric ~/.../sensu-go/types (perf/optimize-check-unmarshal $%)]$ go test -bench=. -benchmem goos: linux goarch: amd64 pkg: github.com/sensu/sensu-go/types BenchmarkCheckRequestMarshal-4 300000 3520 ns/op 3416 B/op 29 allocs/op BenchmarkCheckRequestUnmarshal-4 100000 14693 ns/op 5715 B/op 185 allocs/op BenchmarkCheckConfigMarshal-4 500000 2204 ns/op 1848 B/op 18 allocs/op BenchmarkCheckConfigUnmarshal-4 200000 10317 ns/op 4046 B/op 157 allocs/op PASS ok github.com/sensu/sensu-go/types 6.508s Before: [15:32 eric ~/.../types/dynamic ((6cfa8e40...) $%)]$ go test -bench=. -benchmem goos: linux goarch: amd64 pkg: github.com/sensu/sensu-go/types/dynamic BenchmarkQueryGovaluateSimple-4 1000000 1322 ns/op 320 B/op 18 allocs/op BenchmarkQueryGovaluateComplex-4 300000 4431 ns/op 1160 B/op 66 allocs/op BenchmarkUnmarshal-4 200000 9686 ns/op 7656 B/op 82 allocs/op BenchmarkMarshal-4 300000 4783 ns/op 6528 B/op 43 allocs/op PASS ok github.com/sensu/sensu-go/types/dynamic 6.277s After: [15:32 eric ~/.../types/dynamic (perf/optimize-check-unmarshal $%)]$ go test -bench=. -benchmem goos: linux goarch: amd64 pkg: github.com/sensu/sensu-go/types/dynamic BenchmarkQueryGovaluateSimple-4 1000000 1311 ns/op 320 B/op 18 allocs/op BenchmarkQueryGovaluateComplex-4 300000 4334 ns/op 1160 B/op 66 allocs/op BenchmarkUnmarshal-4 500000 2752 ns/op 1163 B/op 48 allocs/op BenchmarkMarshal-4 500000 2402 ns/op 5647 B/op 38 allocs/op PASS ok github.com/sensu/sensu-go/types/dynamic 5.337s Signed-off-by: Eric Chlebek <eric@sensu.io>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
) | ||
|
||
func init() { | ||
msgPool.New = func() interface{} { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TIL sync.Pool - that's pretty neat.
What is this change?
Why is this change necessary?
Unlike dynamic marshaling, dynamic unmarshaling cannot be avoided. Therefore, we must take steps to optimize it as much as possible to avoid performance problems under load. Dynamic unmarshaling is necessary for every CheckRequest, and cannot be avoided in normal operation. These operations generate many allocations, leading to high GC pressure. This commit substantially reduces these allocations to a more manageable level.
Does your change need a Changelog entry?
Yes