Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Constructor Time Resulting In Mixed Results #1304
Re-running using the default config. Results are given in the same order as the original post:
@adamsitnik I was getting the same results using the default config in my initial tests, but then tried different things and dropped the number of iterations to 1, since I could see that I can successfully replicate the results under the default config each time, and also needed to save time. I should have been more clear in my initial posting, sorry about that.
@luckerby thank you for the full explanation.
This is one of the reasons why BDN runs every benchmark in a stand-alone process - to make sure that things like gen budgets and self-tuning nature of GC does not affect other benchmarks.
But when it comes to things allocated in ctor/setup and the way they affect the benchmark iterations - there is very little we can do.
@adamsitnik - I was thinking about some "look-at-this-tricky-sample-where-you-wouldn't-expect-these-strange-results" example mentioned somewhere in the docs. So people that benchmark and get "wrong" results because of the GC don't quickly point their finger at BDN.
A counter argument could be that if one uses BDN, it's kind of implied that she/he has some connection with the notion of performance and fine-tuning in general, which means in turn that they should already have an idea about the GC in the .NET world. And if they don't know all the nitty-gritty details, at least they're aware of the "knowns unknowns".
I was doing this - ironically - as part of looking into how much time GC takes for a specific scenario involving