Integration with test frameworks

[snoyberg]

It would be great if doctest could be easily run from existing test frameworks. Based on our email discussion[1], it sounds like the best way to do this would be to provide a function:

getDocTests :: [Flag] -> [FilePath] -> IO [Test]

I'm assuming that "Test" here would be from HUnit. Two additional features I would like to ask for are:

• Automatically adding GHC options based on LANGUAGE pragmas on files.
• Determining the [FilePath] argument based on the .cabal file.

Thanks

[1] http://www.haskell.org/pipermail/haskell-cafe/2011-April/090827.html

[sol]

I think LANGUAGE pragmas on files should already work. But we have to take care of additional options given in the .cabal file. Or did I get something wrong?

[snoyberg]

It does not appear to be working for me. Try out the following file:

{-# LANGUAGE QuasiQuotes #-}
module Test where
import Text.Hamlet
import Data.ByteString.Lazy.Char8 (unpack)

-- |
-- >>> unpack $ renderHtml [hamlet|test|]
-- "test"
test :: Int
test = undefined

When run with --optghc=-XQuasiQuotes, it works perfectly. Without, it says:

### Failure in: 0:Test.hs                 
expression `unpack $ renderHtml [hamlet|test|]'
expected: ["\"test\""]
 but got: ["","<interactive>:1:34: parse error on input `]'"]
Cases: 1  Tried: 1  Errors: 0  Failures: 1

[sol]

Sorry for the late response, I had to thoroughly think about it to get it (hopefully) right.

Firstly you are right. I only had the following in mind, which of course works:

{-# LANGUAGE QuasiQuotes #-}

module Test where
import Text.Hamlet
import Data.ByteString.Lazy.Char8 (unpack)

-- |
-- >>> test
-- "test"
test :: String
test = unpack $ renderHtml [hamlet|test|]

doctest behaves like ghci here: Language pragmas on modules are only active for that module and not for the interactive session (and hence doctest). In contrast language options passed with -X are active for all modules (and hence the interactive session, and doctest).

I think, passing module pragmas with -X does not work in general. At least I found a contradiction. Let's assume we have two modules, Foo and Bar:

{-- Foo.hs ----------------------------------}
{-# LANGUAGE Haskell98 #-}
module Foo where
import Bar
foo (n + 1) = n

{-- Bar.hs ----------------------------------}
module Bar where
data S

The following works as expected:
$ echo 'foo 5' | ghci -v0 Foo.hs
4

But passing the language pragma from Foo.hs with -X does not work:
$ echo 'foo 5' | ghci -v0 -XHaskell98 Foo.hs

Bar.hs:3:1:
    `S' has no constructors (-XEmptyDataDecls permits this)
    In the data type declaration for `S'

<interactive>:1:1: Not in scope: `foo'

Right now I see nothing that we can do to make your use case more convenient without breaking other code. Any ideas?

[sol]

As Daniel Fischer pointed out, it is possible to change language flags at runtime with :set -Xlanguage. So we can solve this, the following works:
$ echo -e ':set -XHaskell98\nfoo 5' | ghci -v0 Foo.hs
4

[sakari]

I started exposing doctest as a library at https://github.com/sakari/doctest-haskell/tree/sakari%2Fdoctest-lib any comments on the work in progress are appreciated. The idea is just to get the bare bones functionality working together with https://github.com/sakari/test-framework-doctest i.e., I will not work on cabal integration or language pragma setting.

Maybe those could be separated into new issues?

[sol]

Hey, nice to see work on this!

Yes, it is probably a good idea to have separate issues for both cabal integration and language pragma settings.

[sol]

This has been mostly implemented by sakari. We have #6 and #7 for the remaining issues.