Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

producer: increase performance of sampling rate lookup #264

Merged
merged 1 commit into from
Dec 29, 2023

Conversation

vincentbernat
Copy link
Contributor

Formatting a string with version and domain when looking up sampling rate hurts performance. Instead, use a dedicated structure for key. It also reduces the pressure on the GC.

Before:

BenchmarkDecodeEncodeNetflow/with_encoding-12             158208              8611 ns/op            8178 B/op        130 allocs/op
BenchmarkDecodeEncodeNetflow/without_encoding-12          162832              7559 ns/op            8178 B/op        130 allocs/op

After:

BenchmarkDecodeEncodeNetflow/with_encoding-12             138242              8132 ns/op            8176 B/op        129 allocs/op
BenchmarkDecodeEncodeNetflow/without_encoding-12          162312              7306 ns/op            8176 B/op        129 allocs/op

Formatting a string with version and domain when looking up sampling
rate hurts performance. Instead, use a dedicated structure for key. It
also reduces the pressure on the GC.

Before:

BenchmarkDecodeEncodeNetflow/with_encoding-12             158208              8611 ns/op            8178 B/op        130 allocs/op
BenchmarkDecodeEncodeNetflow/without_encoding-12          162832              7559 ns/op            8178 B/op        130 allocs/op

After:

BenchmarkDecodeEncodeNetflow/with_encoding-12             138242              8132 ns/op            8176 B/op        129 allocs/op
BenchmarkDecodeEncodeNetflow/without_encoding-12          162312              7306 ns/op            8176 B/op        129 allocs/op
@lspgn lspgn added ipfix Related to IPFIX protocol performance Related to application performance labels Dec 25, 2023
@lspgn
Copy link
Member

lspgn commented Dec 25, 2023

Thank you! Could you add the benchmarks?

Not seeing a major difference in perfs though:

BenchmarkTemplatesAdd
    templates_test.go:15: Creating 1 templates
    templates_test.go:15: Creating 100 templates
    templates_test.go:15: Creating 10000 templates
    templates_test.go:15: Creating 1000000 templates
    templates_test.go:15: Creating 6573939 templates
BenchmarkTemplatesAdd-4      	 6573939	       171.8 ns/op
BenchmarkTemplatesAddGet
    templates_test.go:22: Adding 1000 templates
    templates_test.go:24: Getting 1 templates
    templates_test.go:22: Adding 1000 templates
    templates_test.go:24: Getting 100 templates
    templates_test.go:22: Adding 1000 templates
    templates_test.go:24: Getting 10000 templates
    templates_test.go:22: Adding 1000 templates
    templates_test.go:24: Getting 1000000 templates
    templates_test.go:22: Adding 1000 templates
    templates_test.go:24: Getting 24474410 templates
BenchmarkTemplatesAddGet-4   	24474410	        47.66 ns/op

Vs

BenchmarkTemplatesAdd
    templates_test.go:15: Creating 1 templates
    templates_test.go:15: Creating 100 templates
    templates_test.go:15: Creating 10000 templates
    templates_test.go:15: Creating 1000000 templates
    templates_test.go:15: Creating 4606766 templates
    templates_test.go:15: Creating 7184278 templates
BenchmarkTemplatesAdd-4      	 7184278	       168.3 ns/op
BenchmarkTemplatesAddGet
    templates_test.go:22: Adding 1000 templates
    templates_test.go:24: Getting 1 templates
    templates_test.go:22: Adding 1000 templates
    templates_test.go:24: Getting 100 templates
    templates_test.go:22: Adding 1000 templates
    templates_test.go:24: Getting 10000 templates
    templates_test.go:22: Adding 1000 templates
    templates_test.go:24: Getting 1000000 templates
    templates_test.go:22: Adding 1000 templates
    templates_test.go:24: Getting 24551028 templates
BenchmarkTemplatesAddGet-4   	24551028	        46.35 ns/op

@vincentbernat
Copy link
Contributor Author

Well, you get the performance increase too (~2%). Moreover, because there is an allocation less, this reduces GC pressure.

@lspgn
Copy link
Member

lspgn commented Dec 25, 2023

Oh yes, it's a good improvement. Was just hoping to dive into the benchmarks to understand, if easy to add? Otherwise I'll just merge.

@vincentbernat
Copy link
Contributor Author

That was an end-to-end benchmark I have in Akvorado.

@lspgn lspgn merged commit 1a90055 into netsampler:main Dec 29, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ipfix Related to IPFIX protocol performance Related to application performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants