You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To avoid systematic bias that might be caused by operating system caches or hardware/memory properties, the execution order of benchmark runs should be randomized.
This has the additional benefit of producing data points for more benchmarks and it is possible to see trends of the results earlier for long running sets of benchmarks.
There needs to be an option to suppress the randomization, of course.
Also, we should have an estimate of what that means in terms of memory usage when many microbenchmarks are executed (need to keep the data in memory to calculated confidence interval, etc...)
The text was updated successfully, but these errors were encountered:
To avoid systematic bias that might be caused by operating system caches or hardware/memory properties, the execution order of benchmark runs should be randomized.
This has the additional benefit of producing data points for more benchmarks and it is possible to see trends of the results earlier for long running sets of benchmarks.
There needs to be an option to suppress the randomization, of course.
Also, we should have an estimate of what that means in terms of memory usage when many microbenchmarks are executed (need to keep the data in memory to calculated confidence interval, etc...)
The text was updated successfully, but these errors were encountered: