Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flakey test #1755

Closed
KevinRansom opened this issue Nov 16, 2016 · 1 comment
Closed

Flakey test #1755

KevinRansom opened this issue Nov 16, 2016 · 1 comment

Comments

@KevinRansom
Copy link
Member

This test fails on AppVeyor quite frequently:

1) Failed : FSharp.Core.Unittests.FSharp_Core.Microsoft_FSharp_Control.AsyncModule.OnCancel.RaceBetweenCancellationHandlerAndDisposingHandlerRegistration
[00:53:00]   Expected: True
[00:53:00]   But was:  False
[00:53:00] at <StartupCode$FSharp-Core-Unittests>.$AsyncModule.test@286-22(Unit unitVar0) in C:\projects\visualfsharp-3dtit\src\fsharp\FSharp.Core.Unittests\FSharp.Core\Microsoft.FSharp.Control\AsyncModule.fs:line 299
[00:53:00] at FSharp.Core.Unittests.FSharp_Core.Microsoft_FSharp_Control.AsyncModule.OnCancel.RaceBetweenCancellationHandlerAndDisposingHandlerRegistration

We should make it a little more deterministic.

Kevin

Results (nunit3) saved as C:\projects\visualfsharp-3dtit\tests\TestResults\test-net40-coreunit-results.xml
[00:42:36] Saved errorlevel 0
[00:42:36] powershell -File Upload-Results.ps1 "C:\projects\visualfsharp-3dtit\tests\TestResults\test-net40-coreunit-results.xml"
[00:42:37] Uploading results C:\projects\visualfsharp-3dtit\tests\TestResults\test-net40-coreunit-results.xml to https://ci.appveyor.com/api/testresults/nunit/pqtnmxno8245uulp
[00:42:50] "C:\projects\visualfsharp-3dtit\packages\NUnit.Console.3.0.0\tools\nunit3-console.exe" /framework:V4.0 /result="C:\projects\visualfsharp-3dtit\tests\TestResults\test-portable-coreunit-results.xml;format=nunit3" --output:"C:\projects\visualfsharp-3dtit\tests\TestResults\test-portable-coreunit-output.log"  --err:"C:\projects\visualfsharp-3dtit\tests\TestResults\test-portable-coreunit-errors.log"  /work="C:\projects\visualfsharp-3dtit\release\net40\bin" "C:\projects\visualfsharp-3dtit\release\net40\bin\..\..\portable7\bin\FSharp.Core.Unittests.dll" "C:\projects\visualfsharp-3dtit\release\net40\bin\..\..\portable47\bin\FSharp.Core.Unittests.dll" "C:\projects\visualfsharp-3dtit\release\net40\bin\..\..\portable78\bin\FSharp.Core.Unittests.dll" "C:\projects\visualfsharp-3dtit\release\net40\bin\..\..\portable259\bin\FSharp.Core.Unittests.dll" 
[00:42:50] NUnit Console Runner 3.0.5797 
[00:42:50] Copyright (C) 2015 Charlie Poole
[00:42:50] 
[00:42:50] Runtime Environment
[00:42:50]    OS Version: Microsoft Windows NT 6.3.9600.0
[00:42:50]   CLR Version: 4.0.30319.42000
[00:42:50] 
[00:42:50] Test Files
[00:42:50]     C:\projects\visualfsharp-3dtit\release\net40\bin\..\..\portable7\bin\FSharp.Core.Unittests.dll
[00:42:50]     C:\projects\visualfsharp-3dtit\release\net40\bin\..\..\portable47\bin\FSharp.Core.Unittests.dll
[00:42:50]     C:\projects\visualfsharp-3dtit\release\net40\bin\..\..\portable78\bin\FSharp.Core.Unittests.dll
[00:42:50]     C:\projects\visualfsharp-3dtit\release\net40\bin\..\..\portable259\bin\FSharp.Core.Unittests.dll
[00:42:50] 
[00:53:00] 
[00:53:00] Tests Not Run
[00:53:00] 
[00:53:00] 1) Explicit : FSharp.Core.Unittests.FSharp_Core.Microsoft_FSharp_Collections.CollectionModulesConsistency.unfold is consistent full
[00:53:00] 
[00:53:00] 2) Explicit : FSharp.Core.Unittests.FSharp_Core.Microsoft_FSharp_Control.AsyncModule.Async.AwaitWaitHandle does not leak memory
[00:53:00] 
[00:53:00] 3) Explicit : FSharp.Core.Unittests.FSharp_Core.Microsoft_FSharp_Control.AsyncModule.Async.AwaitWaitHandle does not leak memory
[00:53:00] 
[00:53:00] 4) Explicit : FSharp.Core.Unittests.FSharp_Core.Microsoft_FSharp_Collections.CollectionModulesConsistency.unfold is consistent full
[00:53:00] 
[00:53:00] 5) Explicit : FSharp.Core.Unittests.FSharp_Core.Microsoft_FSharp_Control.AsyncModule.Async.AwaitWaitHandle does not leak memory
[00:53:00] 
[00:53:00] 6) Explicit : FSharp.Core.Unittests.FSharp_Core.Microsoft_FSharp_Collections.CollectionModulesConsistency.unfold is consistent full
[00:53:00] 
[00:53:00] 7) Explicit : FSharp.Core.Unittests.FSharp_Core.Microsoft_FSharp_Control.AsyncModule.Async.AwaitWaitHandle does not leak memory
[00:53:00] 
[00:53:00] Errors and Failures
[00:53:00] 
[00:53:00] 1) Failed : FSharp.Core.Unittests.FSharp_Core.Microsoft_FSharp_Control.AsyncModule.OnCancel.RaceBetweenCancellationHandlerAndDisposingHandlerRegistration
[00:53:00]   Expected: True
[00:53:00]   But was:  False
[00:53:00] at <StartupCode$FSharp-Core-Unittests>.$AsyncModule.test@286-22(Unit unitVar0) in C:\projects\visualfsharp-3dtit\src\fsharp\FSharp.Core.Unittests\FSharp.Core\Microsoft.FSharp.Control\AsyncModule.fs:line 299
[00:53:00] at FSharp.Core.Unittests.FSharp_Core.Microsoft_FSharp_Control.AsyncModule.OnCancel.RaceBetweenCancellationHandlerAndDisposingHandlerRegistration() in C:\projects\visualfsharp-3dtit\src\fsharp\FSharp.Core.Unittests\FSharp.Core\Microsoft.FSharp.Control\AsyncModule.fs:line 300
[00:53:00] 
[00:53:00] Run Settings
[00:53:00]     RuntimeFramework: V4.0
[00:53:00]     WorkDirectory: C:\projects\visualfsharp-3dtit\release\net40\bin
[00:53:00]     NumberOfTestWorkers: 2
[00:53:00] 
[00:53:00] Test Run Summary
[00:53:00]     Overall result: Failed
[00:53:00]    Tests run: 20850, Passed: 20849, Errors: 0, Failures: 1, Inconclusive: 0
[00:53:00]      Not run: 7, Invalid: 0, Ignored: 0, Explicit: 7, Skipped: 0
[00:53:00]   Start time: 2016-11-16 20:17:44Z
[00:53:00]     End time: 2016-11-16 20:27:47Z
[00:53:00]     Duration: 602.859 seconds
[00:53:00]
This was referenced Nov 19, 2016
Merged
Merged
KevinRansom pushed a commit that referenced this issue Nov 22, 2016
@kurtschelfthout @KevinRansom
Make async test less flakey. (#1794)
bfd9bc6 
* Make async test less flakey.

See #1755. Instead of a ref cell with a lock, we use a ManualResetEvent
so that the sleep 100ms - which I assume is the reason for the
flakiness, this is not proven - can be replaced with a generous timeout,
while the test can still complete in reasonable time.

* Make sure cancel handler is registered.
@KevinRansom
Copy link
Member Author

This is fixed by this: #1794

Many thanks to @kurtschelfthout

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@KevinRansom