New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The test suite is not deterministic #2

Open
mrkkrp opened this Issue May 10, 2018 · 1 comment

Comments

Projects
None yet
2 participants
@mrkkrp

mrkkrp commented May 10, 2018

The test suite is not deterministic and contributes to flakiness of Stackage HEAD project. For example, with the same build plan and the some docker image, sometimes we get this:

> /tmp/stackage-build2089/fsnotify-conduit-0.1.0.0$ ghc -clear-package-db -global-package-db -package-db=/home/circleci/project/builds/nightly/pkgdb Setup
> /tmp/stackage-build2089/fsnotify-conduit-0.1.0.0$ ./Setup configure --enable-tests --package-db=clear --package-db=global --package-db=/home/circleci/project/builds/nightly/pkgdb --libdir=/home/circleci/project/builds/nightly/lib --bindir=/home/circleci/project/builds/nightly/bin --datadir=/home/circleci/project/builds/nightly/share --libexecdir=/home/circleci/project/builds/nightly/libexec --sysconfdir=/home/circleci/project/builds/nightly/etc --docdir=/home/circleci/project/builds/nightly/doc/fsnotify-conduit-0.1.0.0 --htmldir=/home/circleci/project/builds/nightly/doc/fsnotify-conduit-0.1.0.0 --haddockdir=/home/circleci/project/builds/nightly/doc/fsnotify-conduit-0.1.0.0
Configuring fsnotify-conduit-0.1.0.0...
> /tmp/stackage-build2089/fsnotify-conduit-0.1.0.0$ ghc -clear-package-db -global-package-db -package-db=/home/circleci/project/builds/nightly/pkgdb Setup
> /tmp/stackage-build2089/fsnotify-conduit-0.1.0.0$ ./Setup build
Preprocessing library for fsnotify-conduit-0.1.0.0..
Building library for fsnotify-conduit-0.1.0.0..
[1 of 1] Compiling Data.Conduit.FSNotify ( src/Data/Conduit/FSNotify.hs, dist/build/Data/Conduit/FSNotify.o )

src/Data/Conduit/FSNotify.hs:104:22: warning: [-Wdeprecations]
    In the use of type constructor or class ‘Producer’
    (imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
    Deprecated: "Use ConduitT directly"
    |
104 |                   -> Producer m FS.Event
    |                      ^^^^^^^^
Preprocessing test suite 'fsnotify-conduit-test' for fsnotify-conduit-0.1.0.0..
Building test suite 'fsnotify-conduit-test' for fsnotify-conduit-0.1.0.0..
[1 of 2] Compiling Data.Conduit.FSNotifySpec ( test/Data/Conduit/FSNotifySpec.hs, dist/build/fsnotify-conduit-test/fsnotify-conduit-test-tmp/Data/Conduit/FSNotifySpec.o )

test/Data/Conduit/FSNotifySpec.hs:31:39: warning: [-Wdeprecations]
    In the use of ‘$$’
    (imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
    Deprecated: "Use runConduit and .|"
   |
31 |                                       $$ CL.mapM_ (liftIO . writeChan chan)) <|>
   |                                       ^^
[2 of 2] Compiling Main             ( test/Spec.hs, dist/build/fsnotify-conduit-test/fsnotify-conduit-test-tmp/Main.o )
Linking dist/build/fsnotify-conduit-test/fsnotify-conduit-test ...
> /tmp/stackage-build2089/fsnotify-conduit-0.1.0.0$ dist/build/fsnotify-conduit-test/fsnotify-conduit-test

Data.Conduit.FSNotify
  sourceFileChanges

Finished in 0.5296 seconds
1 example, 0 failures

Other time we're less lucky:

> /tmp/stackage-build2089/fsnotify-conduit-0.1.0.0$ ghc -clear-package-db -global-package-db -package-db=/home/circleci/project/builds/nightly/pkgdb Setup
> /tmp/stackage-build2089/fsnotify-conduit-0.1.0.0$ ./Setup configure --enable-tests --package-db=clear --package-db=global --package-db=/home/circleci/project/builds/nightly/pkgdb --libdir=/home/circleci/project/builds/nightly/lib --bindir=/home/circleci/project/builds/nightly/bin --datadir=/home/circleci/project/builds/nightly/share --libexecdir=/home/circleci/project/builds/nightly/libexec --sysconfdir=/home/circleci/project/builds/nightly/etc --docdir=/home/circleci/project/builds/nightly/doc/fsnotify-conduit-0.1.0.0 --htmldir=/home/circleci/project/builds/nightly/doc/fsnotify-conduit-0.1.0.0 --haddockdir=/home/circleci/project/builds/nightly/doc/fsnotify-conduit-0.1.0.0
Configuring fsnotify-conduit-0.1.0.0...
> /tmp/stackage-build2089/fsnotify-conduit-0.1.0.0$ ghc -clear-package-db -global-package-db -package-db=/home/circleci/project/builds/nightly/pkgdb Setup
> /tmp/stackage-build2089/fsnotify-conduit-0.1.0.0$ ./Setup build
Preprocessing library for fsnotify-conduit-0.1.0.0..
Building library for fsnotify-conduit-0.1.0.0..
[1 of 1] Compiling Data.Conduit.FSNotify ( src/Data/Conduit/FSNotify.hs, dist/build/Data/Conduit/FSNotify.o )

src/Data/Conduit/FSNotify.hs:104:22: warning: [-Wdeprecations]
    In the use of type constructor or class ‘Producer’
    (imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
    Deprecated: "Use ConduitT directly"
    |
104 |                   -> Producer m FS.Event
    |                      ^^^^^^^^
Preprocessing test suite 'fsnotify-conduit-test' for fsnotify-conduit-0.1.0.0..
Building test suite 'fsnotify-conduit-test' for fsnotify-conduit-0.1.0.0..
[1 of 2] Compiling Data.Conduit.FSNotifySpec ( test/Data/Conduit/FSNotifySpec.hs, dist/build/fsnotify-conduit-test/fsnotify-conduit-test-tmp/Data/Conduit/FSNotifySpec.o )

test/Data/Conduit/FSNotifySpec.hs:31:39: warning: [-Wdeprecations]
    In the use of ‘$$’
    (imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
    Deprecated: "Use runConduit and .|"
   |
31 |                                       $$ CL.mapM_ (liftIO . writeChan chan)) <|>
   |                                       ^^
[2 of 2] Compiling Main             ( test/Spec.hs, dist/build/fsnotify-conduit-test/fsnotify-conduit-test-tmp/Main.o )
Linking dist/build/fsnotify-conduit-test/fsnotify-conduit-test ...
> /tmp/stackage-build2089/fsnotify-conduit-0.1.0.0$ dist/build/fsnotify-conduit-test/fsnotify-conduit-test

Data.Conduit.FSNotify
  sourceFileChanges FAILED [1]

Failures:

  test/Data/Conduit/FSNotifySpec.hs:39:17: 
  1) Data.Conduit.FSNotify sourceFileChanges
       expected: "bar"
        but got: "foo"

Randomized with seed 1900622263

Finished in 0.1669 seconds
1 example, 1 failure

I'll probably need to open a PR to disable this test suite somehow on Stackage till it's fixed.

snoyberg added a commit that referenced this issue May 11, 2018

@snoyberg

This comment has been minimized.

Member

snoyberg commented May 13, 2018

I've just uploaded version 0.1.1.0, which should provide a more reliable test suite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment