New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dry mode doesn't work because of the ZeroMeasurementHelper #943

Closed
AndreyAkinshin opened this Issue Nov 1, 2018 · 6 comments

Comments

Projects
None yet
4 participants
@AndreyAkinshin
Member

AndreyAkinshin commented Nov 1, 2018

Unhandled Exception: System.ArgumentOutOfRangeException: n should be >= 1
Parameter name: n
   at BenchmarkDotNet.Mathematics.MathHelper.StudentTwoTail(Double t, Double n)
   at BenchmarkDotNet.Mathematics.MathHelper.StudentOneTail(Double t, Double n)
   at BenchmarkDotNet.Mathematics.StatisticalTesting.StudentTest.IsGreater(Statistics sample, Double value, Threshold threshold)
   at BenchmarkDotNet.Analysers.ZeroMeasurementHelper.CheckZeroMeasurementOneSample(Double[] results, Double threshold)

@AndreyAkinshin AndreyAkinshin added the bug label Nov 1, 2018

@AndreyAkinshin AndreyAkinshin added this to the v0.11.3 milestone Nov 1, 2018

@AndreyAkinshin AndreyAkinshin self-assigned this Nov 1, 2018

AndreyAkinshin added a commit that referenced this issue Nov 1, 2018

@AndreyAkinshin

This comment has been minimized.

Member

AndreyAkinshin commented Nov 1, 2018

Integration tests didn't catch this problem because default analyzers were disabled in BenchmarkTestExecutor.

@adamsitnik

This comment has been minimized.

Member

adamsitnik commented Nov 1, 2018

Integration tests didn't catch this problem because default analyzers were disabled in BenchmarkTestExecutor.

thanks for putting the explanation!

@AndreyAkinshin

This comment has been minimized.

Member

AndreyAkinshin commented Nov 1, 2018

With enabled analysers, we found another problem: https://ci.appveyor.com/project/dotnetfoundation/benchmarkdotnet/builds/19981806/tests

AndreyAkinshin added a commit that referenced this issue Nov 1, 2018

Fix another problem in ZeroMeasurementAnalyser (see #943)
Also fixes the CustomToolchainsAreSupported test
@danmosemsft

This comment has been minimized.

Member

danmosemsft commented Nov 10, 2018

Hello, I hit this, do you have an idea when you will release 0.11.3?

@Rizzen

This comment has been minimized.

Collaborator

Rizzen commented Nov 10, 2018

Hello, I hit this, do you have an idea when you will release 0.11.3?

Hello! You can also use nightly build from bdn-nightly feed.

@danmosemsft

This comment has been minimized.

Member

danmosemsft commented Nov 10, 2018

@Rizzen thanks, I missed that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment