-
-
Notifications
You must be signed in to change notification settings - Fork 266
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Any consideration of having benchmarks? #239
Comments
Should be pretty easy; I'd be willing to toss some together. A few questions: What kind of scope would be appropriate for each test? For example, I could do:
(I could do all of the above pretty easily if desired.) I assume purely single threaded benchmarks would be more relevant for runtime purposes. Would there be any value in internally multithreaded benchmarks? Is there anything else I should keep in mind when setting up the benchmarks to make including them easier? |
Thank you @RossNordby for the quick and prompt response. Yes, all of the above categories looks good and it will help us keep track of individual components as well as broader scenarios.
That's correct.
I don't think so for Microbenchmarks perspective. We can certainly do it in future as a standalone benchmarks.
I think if you can make it runnable using BDN, that should be good enough. If you are ok with it, I would like to include it in |
Alright, there's now a selection of benchmarks: https://github.com/bepu/bepuphysics2/tree/master/DemoBenchmarks I haven't yet included microbenchmarks of the It uses project references as-is, but the prerelease versions of the nuget package should work too.
Sounds good; you have my permission! |
Sweet. Thanks for the quick turnaround. Is there an easy way to tell which of the 3 categories they belong to? Perhaps a README.md in that folder? I will start the process to port them over. |
|
I did a run of these benchmarks (results dotnet/performance#2672 (comment)) and seems like it takes around 45 minutes to complete. Is it possible to trim them down little bit? |
Yup, some have obviously higher value than others- about what time budget are you targeting? I'll see if I can prioritize within that. |
We can target for 15 minutes and see how many we can accomodate. We can adjust it depending on the coverage we get in those 15 minutes. |
d9b1eea brings it to around 12 minutes on my machine. Some benchmarks are merged, others are punted to 'Deep' classes to make it easier to skip. Readme's updated with details; the full suite is still there optionally. |
I tried it on my machine, and I still see it for 40 minutes. Do you see any benchmarks in this list that should have not run?
|
Yup- there should only be 39 benchmarks in the trimmed set, which includes:
The deep dive set should include 123 benchmarks and takes a long time:
Looks like that run included every benchmark unconditionally (128). |
I see it to be 12 minutes.
|
Are there any considerations of having sets of benchmarks using BenchmarkDotNet that can be then included in dotnet/performance repo?
The text was updated successfully, but these errors were encountered: