New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'stack test' not referentially transparent #525

Closed
pharpend opened this Issue Jul 6, 2015 · 6 comments

Comments

Projects
None yet
3 participants
@pharpend
Contributor

pharpend commented Jul 6, 2015

Edit: By the way, I have the latest git version of stack, as of 8:50 AM MDT, 2015-07-06.

Full tree: https://github.com/pharpend/comarkdown.

Here is my lovely test suite, with some compiler errors:

{-# LANGUAGE OverloadedLists #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE Trustworthy #-}

-- This program is free software: you can redistribute it and/or modify
-- it under the terms of the GNU General Public License as published by
-- the Free Software Foundation, either version 3 of the License, or (at
-- your option) any later version.
-- 
-- This program is distributed in the hope that it will be useful, but
-- WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-- General Public License for more details.
-- 
-- You should have received a copy of the GNU General Public License
-- along with this program.  If not, see <http://www.gnu.org/licenses/>.

-- | 
-- Module      : Main
-- Description : The test suite for comarkdown
-- Copyright   : Copyright 2015 Peter Harpending.
-- License     : GPL-3
-- Maintainer  : peter@harpending.org
-- Stability   : experimental
-- Portability : portable

module Main where

import Data.ByteString.Lazy.Char8 (ByteString)
import qualified Data.ByteString.Lazy.Char8 as B
import Text.Comarkdown
import Test.Hspec
import Test.QuickCheck

main :: IO ()
main = hspec $ context "Parsing" $ context "Recognizing bare headers" $ header1Tests

header1Tests :: Spec
header1Tests = context "Header1" $ parallel $ do
  runParseTest "'# something' should be an h1" "# something" $
    (Right [Markdown (Header1 "something")])
  runParseTest "'something\\n====' should be an h1" "something\n===="
    (Right [Markdown (Header1 "something")])
  runParseTest "'something\\n====' should be an h1" "something\n===="
    (Right [Markdown (Header1 "something")])
  runParseTest "'something\\n=' should be an H1" "something\n="
    (Right [Markdown (Header1 "something")])
  runParseTest
    "'# something\\n====' should be an h1 with the '='s as part of the header"
    "# something\n===="
    (Right [Markdown (Header1 "something ====")])
  runParseTest "'# something #' should be an h1" "# something #"
    (Right [Markdown (Header1 "something")])
  runParseTest "'# something ####' should be an h1" "# something ####"
    (Right [Markdown (Header1 "something")])
  specify "'# something' with trailing spaces should be an h1" $
    property $
      \(Nat i) -> do
        let s = mappend "# something" (replicate i ' ')
        parseResult <- runIO $ parse "test" (B.pack s)
        parseResult `shouldBe` Right [Markdown (Header1 "something")]
  specify "'# something #' with trailing spaces should be an h1" $
    property $
      \(Nat i) -> do
        let s = mappend "# something #" (replicate i ' ')
        parseResult <- runIO $ parse "test" (B.pack s)
        parseResult `shouldBe` Right [Markdown (Header1 "something")]
  specify "'# something ' with trailing '#'s should be an h1" $
    property $
      \(Nat i) -> do
        let s = mappend "# something " (replicate i '#')
        parseResult <- runIO $ parse "test" (B.pack s)
        parseResult `shouldBe` Right [Markdown (Header1 "something")]
  specify
    "'# something ' with trailing spaces, trailing '#'s, and more trailing spaces should be an h1" $
    property $
      \((Nat i, Nat j, Nat k)) -> do
        let s = mconcat ["# something ", replicate i ' ', replicate j '#', replicate k ' ']
        parseResult <- runIO $ parse "test" (B.pack s)
        parseResult `shouldBe` Right [Markdown (Header1 "something")]

runParseTest :: String -> ByteString -> Either String Document -> Spec
runParseTest spec bs supposedResult = it spec $ do
  res <- runIO $ parse "test input" bs
  res `shouldBe` supposedResult

newtype Nat = Nat {unNat :: Int}

instance Arbitrary Nat where
  arbitrary = Nat <$> (suchThat arbitrary (>= 0))

If I try to pipe the result of stack test to a pastebin, this is what shows up on the pastebin:

comarkdown-0.1.0.0: test (suite: spec)

Parsing
  Recognizing bare headers
    Header1
      '# something' should be an h1 FAILED [1]
      'something\n====' should be an h1 FAILED [2]
      'something\n====' should be an h1 FAILED [3]
      'something\n=' should be an H1 FAILED [4]
      '# something\n====' should be an h1 with the '='s as part of the header FAILED [5]
      '# something #' should be an h1 FAILED [6]
      '# something ####' should be an h1 FAILED [7]
      '# something' with trailing spaces should be an h1 FAILED [8]

Failures:

  1) Parsing, Recognizing bare headers, Header1, '# something' should be an h1
       uncaught exception: ErrorCall (Prelude.undefined)

  2) Parsing, Recognizing bare headers, Header1, 'something\n====' should be an h1
       uncaught exception: ErrorCall (Prelude.undefined)

  3) Parsing, Recognizing bare headers, Header1, 'something\n====' should be an h1
       uncaught exception: ErrorCall (Prelude.undefined)

  4) Parsing, Recognizing bare headers, Header1, 'something\n=' should be an H1
       uncaught exception: ErrorCall (Prelude.undefined)

  5) Parsing, Recognizing bare headers, Header1, '# something\n====' should be an h1 with the '='s as part of the header
       uncaught exception: ErrorCall (Prelude.undefined)

  6) Parsing, Recognizing bare headers, Header1, '# something #' should be an h1
       uncaught exception: ErrorCall (Prelude.undefined)

  7) Parsing, Recognizing bare headers, Header1, '# something ####' should be an h1
       uncaught exception: ErrorCall (Prelude.undefined)

  8) Parsing, Recognizing bare headers, Header1, '# something' with trailing spaces should be an h1
       uncaught exception: ErrorCall (Prelude.undefined)

Randomized with seed 872476737

Finished in 0.0006 seconds
8 examples, 8 failures
Test suite failure for package comarkdown-0.1.0.0
    spec:  exited with: ExitFailure 1
Logs printed to console

That is, the output of the last successfully built test suite.

If I don't change the test suite, and try to run stack test again, I get the same output.

However, before attempting to pipe the output of stack test to a pastebin, the output was this:

comarkdown-0.1.0.0: build (test)
Preprocessing library comarkdown-0.1.0.0...
In-place registering comarkdown-0.1.0.0...
Preprocessing test suite 'spec' for comarkdown-0.1.0.0...
[1 of 1] Compiling Main             ( spec/Spec.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.2.0/build/spec/spec-tmp/Main.o )

spec/Spec.hs:61:9:
    Couldn't match type ‘IO’
                   with ‘hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM a1’
    Expected type: hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM
                     a1 ()
      Actual type: Expectation
    In a stmt of a 'do' block:
      parseResult `shouldBe` Right [Markdown (Header1 "something")]
    In the expression:
      do { let s = mappend "# something" (replicate i ' ');
           parseResult <- runIO $ parse "test" (B.pack s);
           parseResult `shouldBe` Right [Markdown (Header1 "something")] }
    In the second argument of ‘($)’, namely
      ‘\ (Nat i)
         -> do { let ...;
                 parseResult <- runIO $ parse "test" (B.pack s);
                 .... }’

spec/Spec.hs:67:9:
    Couldn't match type ‘IO’
                   with ‘hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM a2’
    Expected type: hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM
                     a2 ()
      Actual type: Expectation
    In a stmt of a 'do' block:
      parseResult `shouldBe` Right [Markdown (Header1 "something")]
    In the expression:
      do { let s = mappend "# something #" (replicate i ' ');
           parseResult <- runIO $ parse "test" (B.pack s);
           parseResult `shouldBe` Right [Markdown (Header1 "something")] }
    In the second argument of ‘($)’, namely
      ‘\ (Nat i)
         -> do { let ...;
                 parseResult <- runIO $ parse "test" (B.pack s);
                 .... }’

spec/Spec.hs:73:9:
    Couldn't match type ‘IO’
                   with ‘hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM a3’
    Expected type: hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM
                     a3 ()
      Actual type: Expectation
    In a stmt of a 'do' block:
      parseResult `shouldBe` Right [Markdown (Header1 "something")]
    In the expression:
      do { let s = mappend "# something " (replicate i '#');
           parseResult <- runIO $ parse "test" (B.pack s);
           parseResult `shouldBe` Right [Markdown (Header1 "something")] }
    In the second argument of ‘($)’, namely
      ‘\ (Nat i)
         -> do { let ...;
                 parseResult <- runIO $ parse "test" (B.pack s);
                 .... }’

spec/Spec.hs:80:9:
    Couldn't match type ‘IO’
                   with ‘hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM a4’
    Expected type: hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM
                     a4 ()
      Actual type: Expectation
    In a stmt of a 'do' block:
      parseResult `shouldBe` Right [Markdown (Header1 "something")]
    In the expression:
      do { let s = mconcat ...;
           parseResult <- runIO $ parse "test" (B.pack s);
           parseResult `shouldBe` Right [Markdown (Header1 "something")] }
    In the second argument of ‘($)’, namely
      ‘\ ((Nat i, Nat j, Nat k))
         -> do { let ...;
                 parseResult <- runIO $ parse "test" (B.pack s);
                 .... }’

spec/Spec.hs:85:3:
    Couldn't match type ‘IO’
                   with ‘hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM a0’
    Expected type: hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM
                     a0 ()
      Actual type: Expectation
    In a stmt of a 'do' block: res `shouldBe` supposedResult
    In the second argument of ‘($)’, namely
      ‘do { res <- runIO $ parse "test input" bs;
            res `shouldBe` supposedResult }’
    In the expression:
      it spec
      $ do { res <- runIO $ parse "test input" bs;
             res `shouldBe` supposedResult }

--  While building package comarkdown-0.1.0.0 using:
      /usr/bin/runhaskell -package=Cabal-1.22.2.0 -clear-package-db -global-package-db -package-db=/home/pete/.stack/snapshots/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ /tmp/stack16906/Setup.hs --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.2.0/ build test:spec
    Process exited with code: ExitFailure 1
stack test  2.44s user 1.63s system 163% cpu 2.492 total

If I make a trivial change to the test suite, and run stack test again, I get the compiler-errors output.

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Jul 6, 2015

Please see https://github.com/commercialhaskell/stack/blob/master/CONTRIBUTING.md for guidelines on proper bug reporting format, which will make it possible to diagnose this efficiently. In particular, please:

  1. Include --version and --verbose output. The timestamp of the latest version you're using is not sufficient.
  2. Provide step-by-step reproduction, ideally with a minimal reproducing case.
@pharpend

This comment has been minimized.

Contributor

pharpend commented Jul 6, 2015

stack --version: Version 0.1.2.0, Git revision dafc5bb

Minimal steps:

$ git clone git://github.com/pharpend/comarkdown.git
$ cd comarkdown
$ stack test

Observe:

Version 0.1.2.0, Git revision dafc5bb91d470fd4fd9bcca93357db839d952320
2015-07-06 15:14:47.841513: [debug] Checking for project config at: /home/pete/src/scratch/comarkdown/stack.yaml @(stack-0.1.2.0:Stack.Config src/Stack/Config.hs:496:9)
2015-07-06 15:14:47.842016: [debug] Loading project config file stack.yaml @(stack-0.1.2.0:Stack.Config src/Stack/Config.hs:519:13)
2015-07-06 15:14:47.858314: [debug] Run process: ghc --info @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:47.911724: [debug] Run process: ghc --numeric-version @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:47.935185: [debug] Run process: ghc-pkg --no-user-package-db field --simple-output Cabal id @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:47.958263: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:47.980086: [debug] Checking resolver: nightly-2015-07-03 @(stack-0.1.2.0:Stack.Build.Source src/Stack/Build/Source.hs:86:13)
2015-07-06 15:14:48.253884: [debug] Run process: ghc-pkg --global --no-user-package-db dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:48.309469: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /home/pete/.stack/snapshots/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:48.369489: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /home/pete/src/scratch/comarkdown/.stack-work/install/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:48.962009: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:48.982191: [info] comarkdown-0.1.0.0: build (test) @(stack-0.1.2.0:Stack.Build.Execute src/Stack/Build/Execute.hs:490:18)
2015-07-06 15:14:48.982733: [debug] Run process: /usr/bin/runhaskell -package=Cabal-1.22.2.0 -clear-package-db -global-package-db -package-db=/home/pete/.stack/snapshots/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ /tmp/stack24695/Setup.hs --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.2.0/ build test:spec @(stack-0.1.2.0:Stack.Build.Execute src/Stack/Build/Execute.hs:591:13)
2015-07-06 15:14:49.173324: [info] Preprocessing library comarkdown-0.1.0.0... @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:49.640647: [info] In-place registering comarkdown-0.1.0.0... @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:49.684734: [info] Preprocessing test suite 'spec' for comarkdown-0.1.0.0... @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:49.741924: [info] [1 of 1] Compiling Main             ( spec/Spec.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.2.0/build/spec/spec-tmp/Main.o ) @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:50.13207: [warn]  @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:50.132326: [warn] spec/Spec.hs:69:6: @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:50.132392: [warn]     Couldn't match type ‘IO’ @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:50.132445: [warn]                    with ‘hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM a0’ @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:50.132504: [warn]     Expected type: hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:50.132552: [warn]                      a0 () @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:50.132601: [warn]       Actual type: Expectation @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:50.132649: [warn]     In a stmt of a 'do' block: res `shouldBe` supposedResult @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:50.132699: [warn]     In the second argument of ‘($)’, namely @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:50.132746: [warn]       ‘do { res <- runIO $ parse "test input" bs; @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:50.132793: [warn]             res `shouldBe` supposedResult }’ @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:50.132852: [warn]     In the expression: @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:50.132895: [warn]       it spec @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:50.13294: [warn]       $ do { res <- runIO $ parse "test input" bs; @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:14:50.132986: [warn]              res `shouldBe` supposedResult } @(<unknown>:<unknown> <unknown>:0:0)

--  While building package comarkdown-0.1.0.0 using:
      /usr/bin/runhaskell -package=Cabal-1.22.2.0 -clear-package-db -global-package-db -package-db=/home/pete/.stack/snapshots/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ /tmp/stack24695/Setup.hs --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.2.0/ build test:spec
    Process exited with code: ExitFailure 1
stack test --verbose  2.75s user 1.35s system 171% cpu 2.390 total

Run the command again:

$ stack test --verbose

This produces this output:

Version 0.1.2.0, Git revision dafc5bb91d470fd4fd9bcca93357db839d952320
2015-07-06 15:15:25.585158: [debug] Checking for project config at: /home/pete/src/scratch/comarkdown/stack.yaml @(stack-0.1.2.0:Stack.Config src/Stack/Config.hs:496:9)
2015-07-06 15:15:25.585662: [debug] Loading project config file stack.yaml @(stack-0.1.2.0:Stack.Config src/Stack/Config.hs:519:13)
2015-07-06 15:15:25.603144: [debug] Run process: ghc --info @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:25.665702: [debug] Run process: ghc --numeric-version @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:25.694817: [debug] Run process: ghc-pkg --no-user-package-db field --simple-output Cabal id @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:25.722815: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:25.748573: [debug] Checking resolver: nightly-2015-07-03 @(stack-0.1.2.0:Stack.Build.Source src/Stack/Build/Source.hs:86:13)
2015-07-06 15:15:26.004768: [debug] Run process: ghc-pkg --global --no-user-package-db dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:26.055598: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /home/pete/.stack/snapshots/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:26.102769: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /home/pete/src/scratch/comarkdown/.stack-work/install/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:26.666756: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:26.694196: [error] Test suite spec executable not found for comarkdown @(stack-0.1.2.0:Stack.Build.Execute src/Stack/Build/Execute.hs:792:25)
Test suite failure for package comarkdown-0.1.0.0
    spec:  executable not found
Logs printed to console

If you keep running stack test --verbose, you will continue getting
the same output as above

Make this trivial change:

diff --git a/spec/Spec.hs b/spec/Spec.hs
index c824a4c..934967b 100644
--- a/spec/Spec.hs
+++ b/spec/Spec.hs
@@ -1,4 +1,5 @@
 {-# LANGUAGE OverloadedLists #-}
+
 {-# LANGUAGE OverloadedStrings #-}
 {-# LANGUAGE Trustworthy #-}

Run:

$ stack test --verbose

Observe compiler-error output from before:

Version 0.1.2.0, Git revision dafc5bb91d470fd4fd9bcca93357db839d952320
2015-07-06 15:16:59.072079: [debug] Checking for project config at: /home/pete/src/scratch/comarkdown/stack.yaml @(stack-0.1.2.0:Stack.Config src/Stack/Config.hs:496:9)
2015-07-06 15:16:59.072525: [debug] Loading project config file stack.yaml @(stack-0.1.2.0:Stack.Config src/Stack/Config.hs:519:13)
2015-07-06 15:16:59.08253: [debug] Run process: ghc --info @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:16:59.139785: [debug] Run process: ghc --numeric-version @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:16:59.160003: [debug] Run process: ghc-pkg --no-user-package-db field --simple-output Cabal id @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:16:59.180799: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:16:59.201542: [debug] Checking resolver: nightly-2015-07-03 @(stack-0.1.2.0:Stack.Build.Source src/Stack/Build/Source.hs:86:13)
2015-07-06 15:16:59.447493: [debug] Run process: ghc-pkg --global --no-user-package-db dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:16:59.491833: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /home/pete/.stack/snapshots/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:16:59.551079: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /home/pete/src/scratch/comarkdown/.stack-work/install/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:17:00.125191: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:17:00.145395: [info] comarkdown-0.1.0.0: build (test) @(stack-0.1.2.0:Stack.Build.Execute src/Stack/Build/Execute.hs:490:18)
2015-07-06 15:17:00.145929: [debug] Run process: /usr/bin/runhaskell -package=Cabal-1.22.2.0 -clear-package-db -global-package-db -package-db=/home/pete/.stack/snapshots/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ /tmp/stack27412/Setup.hs --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.2.0/ build test:spec @(stack-0.1.2.0:Stack.Build.Execute src/Stack/Build/Execute.hs:591:13)
2015-07-06 15:17:00.347542: [info] Preprocessing library comarkdown-0.1.0.0... @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:00.798549: [info] In-place registering comarkdown-0.1.0.0... @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:00.848864: [info] Preprocessing test suite 'spec' for comarkdown-0.1.0.0... @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:00.913114: [info] [1 of 1] Compiling Main             ( spec/Spec.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.2.0/build/spec/spec-tmp/Main.o ) @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:01.312219: [warn]  @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:01.312593: [warn] spec/Spec.hs:70:6: @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:01.312721: [warn]     Couldn't match type ‘IO’ @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:01.312793: [warn]                    with ‘hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM a0’ @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:01.312862: [warn]     Expected type: hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:01.312927: [warn]                      a0 () @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:01.312996: [warn]       Actual type: Expectation @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:01.313056: [warn]     In a stmt of a 'do' block: res `shouldBe` supposedResult @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:01.313115: [warn]     In the second argument of ‘($)’, namely @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:01.313171: [warn]       ‘do { res <- runIO $ parse "test input" bs; @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:01.313231: [warn]             res `shouldBe` supposedResult }’ @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:01.31329: [warn]     In the expression: @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:01.313339: [warn]       it spec @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:01.313385: [warn]       $ do { res <- runIO $ parse "test input" bs; @(<unknown>:<unknown> <unknown>:0:0)
2015-07-06 15:17:01.313432: [warn]              res `shouldBe` supposedResult } @(<unknown>:<unknown> <unknown>:0:0)

--  While building package comarkdown-0.1.0.0 using:
      /usr/bin/runhaskell -package=Cabal-1.22.2.0 -clear-package-db -global-package-db -package-db=/home/pete/.stack/snapshots/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ /tmp/stack27412/Setup.hs --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.2.0/ build test:spec
    Process exited with code: ExitFailure 1
stack test --verbose  2.40s user 1.57s system 171% cpu 2.317 total

I guess this is an issue of referential transparency. Apologies, I should know how to write a good bug report by now =p

@pharpend pharpend changed the title from stack test not recompiling tests when piping to a non-terminal to 'stack test' not referentially transparent Jul 6, 2015

snoyberg added a commit that referenced this issue Jul 6, 2015

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Jul 6, 2015

Can you check master and see if that resolves it?

@pharpend

This comment has been minimized.

Contributor

pharpend commented Jul 6, 2015

That appears to have fixed it! Thank you, @snoyberg!

@pharpend pharpend closed this Jul 6, 2015

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Jul 6, 2015

Thank you. That bug has popped in and out of existence multiple times, I'm
glad it's properly fixed now.

On Mon, Jul 6, 2015, 9:30 AM Peter Harpending notifications@github.com
wrote:

That appears to have fixed it! Thank you, @snoyberg
https://github.com/snoyberg!


Reply to this email directly or view it on GitHub
#525 (comment)
.

@DanBurton

This comment has been minimized.

Contributor

DanBurton commented Jul 6, 2015

@(: :0:0)

This looks familiar. I presume this indicates some sort of error that
we've seen before?

On Monday, July 6, 2015, Peter Harpending notifications@github.com wrote:

stack --version: Version 0.1.2.0, Git revision dafc5bb
dafc5bb

Minimal steps:

$ git clone git://github.com/pharpend/comarkdown.git
$ cd comarkdown
$ stack test

Observe:

Version 0.1.2.0, Git revision dafc5bb
2015-07-06 15:14:47.841513: [debug] Checking for project config at: /home/pete/src/scratch/comarkdown/stack.yaml @(stack-0.1.2.0:Stack.Config src/Stack/Config.hs:496:9)
2015-07-06 15:14:47.842016: [debug] Loading project config file stack.yaml @(stack-0.1.2.0:Stack.Config src/Stack/Config.hs:519:13)
2015-07-06 15:14:47.858314: [debug] Run process: ghc --info @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:47.911724: [debug] Run process: ghc --numeric-version @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:47.935185: [debug] Run process: ghc-pkg --no-user-package-db field --simple-output Cabal id @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:47.958263: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:47.980086: [debug] Checking resolver: nightly-2015-07-03 @(stack-0.1.2.0:Stack.Build.Source src/Stack/Build/Source.hs:86:13)
2015-07-06 15:14:48.253884: [debug] Run process: ghc-pkg --global --no-user-package-db dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:48.309469: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /home/pete/.stack/snapshots/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:48.369489: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /home/pete/src/scratch/comarkdown/.stack-work/install/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:48.962009: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:14:48.982191: [info] comarkdown-0.1.0.0: build (test) @(stack-0.1.2.0:Stack.Build.Execute src/Stack/Build/Execute.hs:490:18)
2015-07-06 15:14:48.982733: [debug] Run process: /usr/bin/runhaskell -package=Cabal-1.22.2.0 -clear-package-db -global-package-db -package-db=/home/pete/.stack/snapshots/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ /tmp/stack24695/Setup.hs --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.2.0/ build test:spec @(stack-0.1.2.0:Stack.Build.Execute src/Stack/Build/Execute.hs:591:13)
2015-07-06 15:14:49.173324: [info] Preprocessing library comarkdown-0.1.0.0... @(: :0:0)
2015-07-06 15:14:49.640647: [info] In-place registering comarkdown-0.1.0.0... @(: :0:0)
2015-07-06 15:14:49.684734: [info] Preprocessing test suite 'spec' for comarkdown-0.1.0.0... @(: :0:0)
2015-07-06 15:14:49.741924: [info] [1 of 1] Compiling Main ( spec/Spec.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.2.0/build/spec/spec-tmp/Main.o ) @(: :0:0)
2015-07-06 15:14:50.13207: [warn] @(: :0:0)
2015-07-06 15:14:50.132326: [warn] spec/Spec.hs:69:6: @(: :0:0)
2015-07-06 15:14:50.132392: [warn] Couldn't match type ‘IO’ @(: :0:0)
2015-07-06 15:14:50.132445: [warn] with ‘hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM a0’ @(: :0:0)
2015-07-06 15:14:50.132504: [warn] Expected type: hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM @(: :0:0)
2015-07-06 15:14:50.132552: [warn] a0 () @(: :0:0)
2015-07-06 15:14:50.132601: [warn] Actual type: Expectation @(: :0:0)
2015-07-06 15:14:50.132649: [warn] In a stmt of a 'do' block: res shouldBe supposedResult @(: :0:0)
2015-07-06 15:14:50.132699: [warn] In the second argument of ‘($)’, namely @(: :0:0)
2015-07-06 15:14:50.132746: [warn] ‘do { res <- runIO $ parse "test input" bs; @(: :0:0)
2015-07-06 15:14:50.132793: [warn] res shouldBe supposedResult }’ @(: :0:0)
2015-07-06 15:14:50.132852: [warn] In the expression: @(: :0:0)
2015-07-06 15:14:50.132895: [warn] it spec @(: :0:0)
2015-07-06 15:14:50.13294: [warn] $ do { res <- runIO $ parse "test input" bs; @(: :0:0)
2015-07-06 15:14:50.132986: [warn] res shouldBe supposedResult } @(: :0:0)

-- While building package comarkdown-0.1.0.0 using:
/usr/bin/runhaskell -package=Cabal-1.22.2.0 -clear-package-db -global-package-db -package-db=/home/pete/.stack/snapshots/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ /tmp/stack24695/Setup.hs --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.2.0/ build test:spec
Process exited with code: ExitFailure 1
stack test --verbose 2.75s user 1.35s system 171% cpu 2.390 total

Run the command again:

$ stack test --verbose

This produces this output:

Version 0.1.2.0, Git revision dafc5bb
2015-07-06 15:15:25.585158: [debug] Checking for project config at: /home/pete/src/scratch/comarkdown/stack.yaml @(stack-0.1.2.0:Stack.Config src/Stack/Config.hs:496:9)
2015-07-06 15:15:25.585662: [debug] Loading project config file stack.yaml @(stack-0.1.2.0:Stack.Config src/Stack/Config.hs:519:13)
2015-07-06 15:15:25.603144: [debug] Run process: ghc --info @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:25.665702: [debug] Run process: ghc --numeric-version @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:25.694817: [debug] Run process: ghc-pkg --no-user-package-db field --simple-output Cabal id @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:25.722815: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:25.748573: [debug] Checking resolver: nightly-2015-07-03 @(stack-0.1.2.0:Stack.Build.Source src/Stack/Build/Source.hs:86:13)
2015-07-06 15:15:26.004768: [debug] Run process: ghc-pkg --global --no-user-package-db dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:26.055598: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /home/pete/.stack/snapshots/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:26.102769: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /home/pete/src/scratch/comarkdown/.stack-work/install/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:26.666756: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:15:26.694196: [error] Test suite spec executable not found for comarkdown @(stack-0.1.2.0:Stack.Build.Execute src/Stack/Build/Execute.hs:792:25)
Test suite failure for package comarkdown-0.1.0.0
spec: executable not found
Logs printed to console

If you keep running stack test --verbose, you will continue getting
the same output as above

Make this trivial change:

diff --git a/spec/Spec.hs b/spec/Spec.hs
index c824a4c..934967b 100644--- a/spec/Spec.hs+++ b/spec/Spec.hs@@ -1,4 +1,5 @@
{-# LANGUAGE OverloadedLists #-}+
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE Trustworthy #-}

Run:

$ stack test --verbose

Observe compiler-error output from before:

Version 0.1.2.0, Git revision dafc5bb
2015-07-06 15:16:59.072079: [debug] Checking for project config at: /home/pete/src/scratch/comarkdown/stack.yaml @(stack-0.1.2.0:Stack.Config src/Stack/Config.hs:496:9)
2015-07-06 15:16:59.072525: [debug] Loading project config file stack.yaml @(stack-0.1.2.0:Stack.Config src/Stack/Config.hs:519:13)
2015-07-06 15:16:59.08253: [debug] Run process: ghc --info @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:16:59.139785: [debug] Run process: ghc --numeric-version @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:16:59.160003: [debug] Run process: ghc-pkg --no-user-package-db field --simple-output Cabal id @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:16:59.180799: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:16:59.201542: [debug] Checking resolver: nightly-2015-07-03 @(stack-0.1.2.0:Stack.Build.Source src/Stack/Build/Source.hs:86:13)
2015-07-06 15:16:59.447493: [debug] Run process: ghc-pkg --global --no-user-package-db dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:16:59.491833: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /home/pete/.stack/snapshots/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:16:59.551079: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /home/pete/src/scratch/comarkdown/.stack-work/install/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ dump --expand-pkgroot @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:17:00.125191: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack-0.1.2.0:System.Process.Read src/System/Process/Read.hs:254:3)
2015-07-06 15:17:00.145395: [info] comarkdown-0.1.0.0: build (test) @(stack-0.1.2.0:Stack.Build.Execute src/Stack/Build/Execute.hs:490:18)
2015-07-06 15:17:00.145929: [debug] Run process: /usr/bin/runhaskell -package=Cabal-1.22.2.0 -clear-package-db -global-package-db -package-db=/home/pete/.stack/snapshots/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ /tmp/stack27412/Setup.hs --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.2.0/ build test:spec @(stack-0.1.2.0:Stack.Build.Execute src/Stack/Build/Execute.hs:591:13)
2015-07-06 15:17:00.347542: [info] Preprocessing library comarkdown-0.1.0.0... @(: :0:0)
2015-07-06 15:17:00.798549: [info] In-place registering comarkdown-0.1.0.0... @(: :0:0)
2015-07-06 15:17:00.848864: [info] Preprocessing test suite 'spec' for comarkdown-0.1.0.0... @(: :0:0)
2015-07-06 15:17:00.913114: [info] [1 of 1] Compiling Main ( spec/Spec.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.2.0/build/spec/spec-tmp/Main.o ) @(: :0:0)
2015-07-06 15:17:01.312219: [warn] @(: :0:0)
2015-07-06 15:17:01.312593: [warn] spec/Spec.hs:70:6: @(: :0:0)
2015-07-06 15:17:01.312721: [warn] Couldn't match type ‘IO’ @(: :0:0)
2015-07-06 15:17:01.312793: [warn] with ‘hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM a0’ @(: :0:0)
2015-07-06 15:17:01.312862: [warn] Expected type: hspec-core-2.1.7:Test.Hspec.Core.Spec.Monad.SpecM @(: :0:0)
2015-07-06 15:17:01.312927: [warn] a0 () @(: :0:0)
2015-07-06 15:17:01.312996: [warn] Actual type: Expectation @(: :0:0)
2015-07-06 15:17:01.313056: [warn] In a stmt of a 'do' block: res shouldBe supposedResult @(: :0:0)
2015-07-06 15:17:01.313115: [warn] In the second argument of ‘($)’, namely @(: :0:0)
2015-07-06 15:17:01.313171: [warn] ‘do { res <- runIO $ parse "test input" bs; @(: :0:0)
2015-07-06 15:17:01.313231: [warn] res shouldBe supposedResult }’ @(: :0:0)
2015-07-06 15:17:01.31329: [warn] In the expression: @(: :0:0)
2015-07-06 15:17:01.313339: [warn] it spec @(: :0:0)
2015-07-06 15:17:01.313385: [warn] $ do { res <- runIO $ parse "test input" bs; @(: :0:0)
2015-07-06 15:17:01.313432: [warn] res shouldBe supposedResult } @(: :0:0)

-- While building package comarkdown-0.1.0.0 using:
/usr/bin/runhaskell -package=Cabal-1.22.2.0 -clear-package-db -global-package-db -package-db=/home/pete/.stack/snapshots/x86_64-linux/nightly-2015-07-03/7.10.1/pkgdb/ /tmp/stack27412/Setup.hs --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.2.0/ build test:spec
Process exited with code: ExitFailure 1
stack test --verbose 2.40s user 1.57s system 171% cpu 2.317 total

I guess this is an issue of referential transparency. Apologies, I should
know how to write a good bug report by now =p


Reply to this email directly or view it on GitHub
#525 (comment)
.

-- Dan Burton

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment