Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarks against std vec #2

Closed
makoConstruct opened this issue Aug 13, 2021 · 4 comments · Fixed by #3
Closed

Add benchmarks against std vec #2

makoConstruct opened this issue Aug 13, 2021 · 4 comments · Fixed by #3
Assignees

Comments

@makoConstruct
Copy link
Contributor

makoConstruct commented Aug 13, 2021

(I'll do this now unless you have objections)

@vadixidav
Copy link
Member

Go for it. My focus was mainly on the functionality initially, but, by all means, add any tests and benchmarks you want. I only benchmarked this crate indirectly using hgg, but a direct benchmark would be excellent. Thanks for contributing!

@makoConstruct
Copy link
Contributor Author

resolved #3

Hm, so, to discuss the results, it's getting numbers like

running 6 tests
test test_header_vec_create          ... bench:     246,006 ns/iter (+/- 1,133,418)
test test_header_vec_create_smaller  ... bench:       2,871 ns/iter (+/- 4,370)
test test_header_vec_read            ... bench:         507 ns/iter (+/- 41)
test test_regular_vec_create         ... bench:       3,258 ns/iter (+/- 274)
test test_regular_vec_create_smaller ... bench:         829 ns/iter (+/- 80)
test test_regular_vec_read           ... bench:         509 ns/iter (+/- 14)

Which is not the best. The test_header_vec_create bench's runtime varies between being a factor of 4 to 60 times slower than test_regular_vec_create. The extremeness of that variance could have something to do with bencher using too much memory and with my linux box being deeply stuck in swap right now, so I'm going to reboot and run it again.

@makoConstruct
Copy link
Contributor Author

Okay I rebooted, results far less dire. header vec is about 4.3x slower.

running 6 tests
test test_header_vec_create          ... bench:      10,746 ns/iter (+/- 932)
test test_header_vec_create_smaller  ... bench:       2,562 ns/iter (+/- 55)
test test_header_vec_read            ... bench:         472 ns/iter (+/- 12)
test test_regular_vec_create         ... bench:       2,922 ns/iter (+/- 98)
test test_regular_vec_create_smaller ... bench:         792 ns/iter (+/- 44)
test test_regular_vec_read           ... bench:         470 ns/iter (+/- 12)

@vadixidav
Copy link
Member

Yeah, I will open a tracking issue to see if we can get closer to Vec performance. That could help speed up hgg, and I would be pretty happy about that. Thanks again for the benchmarks and posting the results, that is so helpful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants