stack test --coverage fails with "Failed to find GHC package key"

[madjar]

Hello !

I can't seem to be able to get a coverage reports for my tests. This is on OSX, and I don't have a linux machine handy to try and reproduce it.

Same behaviour for stack 1.1.2, and master.

Thanks a lot!

Steps to reproduce:

cd /tmp
stack new tagada
cd tagada
# Optionally, edit the test file to actually call the library code, but that doesn't change the output.
stack test --coverage
# open the html coverage report

Expected:

Some kind of coverage report

Actual

The coverage report says

HPC Report Generation Error
Failed to find GHC package key for tagada

The logs says

Failed to find GHC package key for tagada
Only one tix file found in /private/tmp/tagada/.stack-work/install/x86_64-osx/lts-6.10/7.10.3/hpc/, so not generating a unified coverage report

stack test --coverage --verbose output

Version 1.1.2 x86_64 hpack-0.14.0
2016-08-01 15:41:50.004125: [debug] Checking for project config at: /private/tmp/tagada/stack.yaml @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.Config src/Stack/Config.hs:811:9)
2016-08-01 15:41:50.004869: [debug] Loading project config file stack.yaml @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.Config src/Stack/Config.hs:829:13)
2016-08-01 15:41:50.005949: [debug] Trying to decode /Users/madjar/.stack/build-plan-cache/x86_64-osx/lts-6.10.cache @(stack_JAcs0EuAGbK5rtqRdoxQaL:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2016-08-01 15:41:50.019662: [debug] Success decoding /Users/madjar/.stack/build-plan-cache/x86_64-osx/lts-6.10.cache @(stack_JAcs0EuAGbK5rtqRdoxQaL:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)
2016-08-01 15:41:50.019794: [debug] Getting system compiler version @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.Setup src/Stack/Setup.hs:341:17)
2016-08-01 15:41:50.020682: [debug] Run process: ghc --info @(stack_JAcs0EuAGbK5rtqRdoxQaL:System.Process.Read src/System/Process/Read.hs:283:3)
2016-08-01 15:41:50.090331: [debug] Asking GHC for its version @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.Setup.Installed src/Stack/Setup/Installed.hs:94:13)
2016-08-01 15:41:50.090429: [debug] Run process: ghc --numeric-version @(stack_JAcs0EuAGbK5rtqRdoxQaL:System.Process.Read src/System/Process/Read.hs:283:3)
2016-08-01 15:41:50.126107: [debug] Getting Cabal package version @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.GhcPkg src/Stack/GhcPkg.hs:165:5)
2016-08-01 15:41:50.126225: [debug] Run process: ghc-pkg --no-user-package-db field --simple-output Cabal version @(stack_JAcs0EuAGbK5rtqRdoxQaL:System.Process.Read src/System/Process/Read.hs:283:3)
2016-08-01 15:41:50.145707: [debug] Resolving package entries @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.Setup src/Stack/Setup.hs:221:5)
2016-08-01 15:41:50.146079: [debug] Getting global package database location @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.GhcPkg src/Stack/GhcPkg.hs:48:5)
2016-08-01 15:41:50.146135: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack_JAcs0EuAGbK5rtqRdoxQaL:System.Process.Read src/System/Process/Read.hs:283:3)
2016-08-01 15:41:50.171137: [debug] Run process: ghc-pkg --global --no-user-package-db dump --expand-pkgroot @(stack_JAcs0EuAGbK5rtqRdoxQaL:System.Process.Read src/System/Process/Read.hs:283:3)
2016-08-01 15:41:50.217530: [debug] Ignoring package Cabal due to wanting version 1.22.8.0 instead of 1.24.0.0 @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-08-01 15:41:50.217674: [debug] Ignoring package haskeline due to wanting version 0.7.2.3 instead of 0.7.2.1 @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-08-01 15:41:50.217743: [debug] Ignoring package terminfo due to wanting version 0.4.0.2 instead of 0.4.0.1 @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-08-01 15:41:50.217840: [debug] Ignoring package Cabal due to wanting version 1.22.8.0 instead of 1.22.5.0 @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-08-01 15:41:50.218055: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /Users/madjar/.stack/snapshots/x86_64-osx/lts-6.10/7.10.3/pkgdb dump --expand-pkgroot @(stack_JAcs0EuAGbK5rtqRdoxQaL:System.Process.Read src/System/Process/Read.hs:283:3)
2016-08-01 15:41:50.240660: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /private/tmp/tagada/.stack-work/install/x86_64-osx/lts-6.10/7.10.3/pkgdb dump --expand-pkgroot @(stack_JAcs0EuAGbK5rtqRdoxQaL:System.Process.Read src/System/Process/Read.hs:283:3)
2016-08-01 15:41:50.264405: [debug] Getting global package database location @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.GhcPkg src/Stack/GhcPkg.hs:48:5)
2016-08-01 15:41:50.264512: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack_JAcs0EuAGbK5rtqRdoxQaL:System.Process.Read src/System/Process/Read.hs:283:3)
2016-08-01 15:41:50.286646: [info] tagada-0.1.0.0: test (suite: tagada-test) @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.Build.Execute src/Stack/Build/Execute.hs:731:23)

2016-08-01 15:41:50.286885: [debug] Creating process: /private/tmp/tagada/.stack-work/dist/x86_64-osx/Cabal-1.24.0.0/build/tagada-test/tagada-test  @(stack_JAcs0EuAGbK5rtqRdoxQaL:System.Process.Run src/System/Process/Run.hs:125:5)
Test suite not yet implemented
2016-08-01 15:41:50.291836: [info]  @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.Build.Execute src/Stack/Build/Execute.hs:1297:52)
2016-08-01 15:41:50.292752: [error] Failed to find GHC package key for tagada @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.Coverage src/Stack/Coverage.hs:119:21)
2016-08-01 15:41:50.293490: [info] Only one tix file found in /private/tmp/tagada/.stack-work/install/x86_64-osx/lts-6.10/7.10.3/hpc/, so not generating a unified coverage report. @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.Coverage src/Stack/Coverage.hs:273:14)
2016-08-01 15:41:50.294042: [info]
An index of the generated HTML coverage reports is available at /private/tmp/tagada/.stack-work/install/x86_64-osx/lts-6.10/7.10.3/hpc/index.html @(stack_JAcs0EuAGbK5rtqRdoxQaL:Stack.Coverage src/Stack/Coverage.hs:369:9)

[ruuda]

I am experiencing the same issue on Linux. After tinkering with various -fhpc and -hpcdir GHC options and the --with-coverage Cabal flag, I had managed to make it work with a Cabal sandbox, but I broke it, and I don’t recall how I made it work.

My Cabal file has the following structure:

name: tagada
version: 0.0.0

library
  ...

executable tagada
  ...
  build-depends: tagada, ...

test-suite spec
  ...
  build-depends: tagada, ...

Stack version: Version 1.1.2 x86_64 hpack-0.14.1
GHC version: The Glorious Glasgow Haskell Compilation System, version 8.0.1

[ruuda]

As a workaround, here’s what worked for me with Cabal:

$ cabal sandbox init
$ cabal install -j --enable-tests
$ cabal configure --enable-tests --enable-coverage
$ cabal build # This step is important, running 'test' immediately will fail.
$ cabal test

I did not need to add -fhpc to the ghc-options anywhere.

[mgsloan]

@ruuda @madjar I believe this is now fixed on the master branch, I'll re-open if not. The issue is that Cabal changed where it was storing this file in 1.24

[ruuda]

Thanks for the quick response! I build Stack 49d96c9 from source, and I am now getting a different error:

Generating coverage report for myapp's test-suite "spec"
Error: The coverage report for myapp's test-suite "spec" did not consider any code. One possible cause of this is if your test-suite builds the library code (see stack issue #1008). It may also indicate a bug in stack or the hpc program. Please report this issue if you think your coverage report should have meaningful results.

My test-suite depends on the library, it does not include the library sources itself. The hs-source-dirs for the library is src and for the test suite it is tests.

[madjar]

I confirm that it works. Thanks a lot!

[mgsloan]

@ruuda Welcome! That's curious. I recommend taking a look at the contents of the tix files located in stack path --local-hpc-root. Are the modules in your library mentioned? What if you force full recompilation via --ghc-options -fforce-recomp --force-dirty?

[ruuda]

@mgsloan A few observations:

When I run

$ stack clean
$ stack build --coverage
$ stack test --coverage

Then I still get the “did not consider any code” message, but it does print statistics, and it does generate an html report. The reports for the test suites (I actually have two test suites) have title “HPC Report Generation Error” with the “did not consider any code” message, but the “combined” package for the “all” test suite does have a full coverage report for my library. If I omit --coverage from stack build, then no reports contain any data.

If I do stack build without --coverage and then stack test --coverage, there are directories in the stack path --local-hpc-root for my test suites, but the tix files in there are empty. (Just Tix [].)

Running stack test with --coverage --ghc-options -fforce-recomp --force-dirty has the same effect as building with --coverage: it also prints the “did not consider any code” message, but it does generate a report nonetheless.

[mgsloan]

Then I still get the “did not consider any code” message, but it does print statistics, and it does generate an html report.

Weird! I can't reproduce this with a basic template.

Look in $(stack path --local-hpc-root) and $(stack path --dist-dir)/hpc and see if anything is obviously wrong. Stack does some tix file munging. I've just pushed a commit which keeps around the file from before munging as a .tix.premunging file.

If I do stack build without --coverage and then stack test --coverage, there are directories in the stack path --local-hpc-root for my test suites, but the tix files in there are empty. (Just Tix [].)

Tracked by #1411

[ruuda]

Weird! I can't reproduce this with a basic template.

I can reproduce even with the steps above, when I change the resolver to nightly-2016-08-06. Is there anything I can do to further diagnose this? I am using Arch Linux with system-installed GHC 8.0.1.

Look in $(stack path --local-hpc-root) and $(stack path --dist-dir)/hpc and see if anything is obviously wrong. Stack does some tix file munging. I've just pushed a commit which keeps around the file from before munging as a .tix.premunging file.

In a fresh stack new project with resolver changed to nightly-2016-08-06, I’ve changed test/Spec.hs to this:

import Lib (someFunc)

main :: IO ()
main = someFunc

To make sure there is some coverage. I ran stack clean and stack test --coverage. Here is the .tix file:

Tix [TixModule "repro-0.1.0.0-lhlx0AjuGfG8cvyrxVOa2/Lib" 1218821433 3 [1,1,1]]

And the .tix.premunging file:

Tix [ TixModule "repro-0.1.0.0-lhlx0AjuGfG8cvyrxVOa2/Lib" 1218821433 3 [1,1,1], TixModule "Main" 2139367740 2 [1,1]]

So Main disappears. That also happens in my actual project.

In the test project I get no html report even when I run stack build --coverage before stack test --coverage. So then I duplicated the test-suite section in the Cabal file, and with a second test suite, it does produce an html report. I created a repository with a minimal example and I included the hpc directories. There are two commits in there, one with a single test suite and one with two.

I’m not sure what I should be looking for in the dist dir, but it contains mix files, one Main.mix that does not mention someFunc and one Lib.mix in a directory, which does mention someFunc.

[mgsloan]

@ruuda Thanks for the repro! Turns out the issue is using --coverage with GHC8. Does my fix work for you?

[ruuda]

@mgsloan: everything works perfectly now. Thanks a lot!