hint test failures #4668

mihaimaruseac · 2019-06-25T14:14:59Z

Cases: 16  Tried: 4  Errors: 0  Failures: 0WontCompile [GhcError {errMsg = "<no location info>: error:\n    Could not load module \8216Data.Map\8217\n    It is a member of the hidden package \8216rerebase-1.3.1.1\8217.\n    You can run \8216:set -package rerebase\8217 to expose it.\n    (Note: this unloads all the modules in the current scope.)\n    It is a member of the hidden package \8216containers-0.6.0.1\8217.\n    You can run \8216:set -package containers\8217 to expose it.\n    (Note: this unloads all the modules in the current scope.)\n    Use -v to see a list of the files searched for."}]
### Error in:   4:qual_import              
user error (())
### Failure in: 9:installed_not_in_scope   
unit-tests/run-unit-tests.hs:319
must be in scope again
Cases: 16  Tried: 16  Errors: 1  Failures: 1
Cases: 1  Tried: 1  Errors: 0  Failures: 0
Cases: 16  Tried: 4  Errors: 0  Failures: 0WontCompile [GhcError {errMsg = "<no location info>: error:\n    Could not load module \8216Data.Map\8217\n    It is a member of the hidden package \8216rerebase-1.3.1.1\8217.\n    You can run \8216:set -package rerebase\8217 to expose it.\n    (Note: this unloads all the modules in the current scope.)\n    It is a member of the hidden package \8216containers-0.6.0.1\8217.\n    You can run \8216:set -package containers\8217 to expose it.\n    (Note: this unloads all the modules in the current scope.)\n    Use -v to see a list of the files searched for."}]
### Error in:   4:qual_import              
user error (())
### Failure in: 9:installed_not_in_scope   
unit-tests/run-unit-tests.hs:319
must be in scope again
Cases: 16  Tried: 16  Errors: 1  Failures: 1
Cases: 1  Tried: 1  Errors: 0  Failures: 0

CC @mvdan

The text was updated successfully, but these errors were encountered:

mvdan · 2019-06-25T14:30:59Z

Sorry, but I no longer maintain hint. See haskell-hint/hint#47. Multiple people have admin access to the repo though, and any one of them can take over the package publishing. It's probably going to be easiest if you file an issue there.

#4670)

gelisam · 2019-07-01T22:50:22Z

I can reproduce the problem on my machine: cabal new-test succeeds but stack test fails. The Travis builds use the old-style cabal test, so it's not clear if this is a regression or if this has always failed with stack. @mihaimaruseac, presumably you found this failure by automatically running the tests on all the packages on stackage, and presumably these automated tests were previously succeeding? Can you tell us what changed between those two tests? Was it a change in stack, in the ghc version, or in the hint version?

gelisam · 2019-07-01T23:00:14Z

I now see that my stack.yaml isn't included in the repo, so I'll include it here for reproducibility's sake and then I'll add one to the repo once this bug is fixed.

resolver: lts-13.18
packages:
- .
extra-deps:
- exceptions-0.10.0

gelisam · 2019-07-07T03:42:19Z

While hint's CI use the old-style cabal test, I have found a related project whose CI uses stack. It's another project of mine, hawk, which uses hint heavily. The CI logs indicate that stack test used to succeed, but today running stack test in that repo fails with a similar error message, saying that some modules are part of a hidden package.

So what has changed between now and then? Perhaps the version of stack? The CI logs do not mention which version of stack was used at the time, but since it was downloaded from https://www.stackage.org/stack/linux-x86_64, presumably it was the version which was the latest at the time (December 2018), so stack 1.9.3. The version I am running now is stack 2.0.0. I will downgrade to stack 1.9.3 to see if the tests pass with that version.

gelisam · 2019-07-07T13:01:25Z

Indeed, the hint and hawk tests both pass with stack 1.9.3 but not with stack 2.0.0.

stack's changelog says the changes should be mostly backwards compatible, so I guess this is a regression in stack?

gelisam · 2019-07-07T13:07:53Z

The stack changelogs do list some user-visible changes. This one seems relevant:

Overriding GHC boot packages results in any other GHC boot packages depending on it being no longer available as a dependency, such packages need to be added explicitly when needed. See #4510.

Data.Map is in the containers package, which is a boot package. However, we've seen the same problem in the hawk test suite with modules from the haskell-awk package, which is definitely not a boot package. So I don't think this is the root cause of the problem, but the fact that we're using a boot package might be compounding the problem.

gelisam · 2019-07-07T13:26:29Z

Here is another entry in the changelog which seems extremely relevant:

Set the GHC_ENVIRONMENT environment variable to specify dependency packages explicitly when running test. This is done to prevent ambiguous module name errors in doctest tests.

Neither hint nor hawk looks at GHC_ENVIRONMENT, but they delegate to the ghc library, which probably does look at GHC_ENVIRONMENT.

hawk uses the HASKELL_PACKAGE_SANDBOXES environment variable, which stack also sets, but hint does not. I wonder what's the difference between those two environment variables?

gelisam · 2019-07-07T15:07:15Z

I found a more comprehensive list of the relevant environment variables. Here are their values.

stack 1.9.3
GHC_PACKAGE_PATH=
  $HINT_ROOT/.stack-work/install/x86_64-osx/lts-13.18/8.6.4/pkgdb
  $HOME/.stack/snapshots/x86_64-osx/lts-13.18/8.6.4/pkgdb
  $HOME/.stack/programs/x86_64-osx/ghc-8.6.4/lib/ghc-8.6.4/package.conf.d
GHC_ENVIRONMENT (unset)
HASKELL_PACKAGE_SANDBOX=
  $HOME/.stack/snapshots/x86_64-osx/lts-13.18/8.6.4/pkgdb
HASKELL_PACKAGE_SANDBOXES=
  $HINT_ROOT/.stack-work/install/x86_64-osx/lts-13.18/8.6.4/pkgdb
  $HOME/.stack/snapshots/x86_64-osx/lts-13.18/8.6.4/pkgdb
HASKELL_DIST_DIR=
  .stack-work/dist/x86_64-osx/Cabal-2.4.0.1
DESTDIR (unset)
GHCRTS (unset)

stack 2.1.1 (turns out my stack 2.0.0 was not an official release)
GHC_PACKAGE_PATH=
  $HINT_ROOT/.stack-work/install/x86_64-osx/2c51859fcfd0b0bbd239f96f11700837286b155df4d5d28072631deb79498000/8.6.4/pkgdb
  $HOME/.stack/snapshots/x86_64-osx/2c51859fcfd0b0bbd239f96f11700837286b155df4d5d28072631deb79498000/8.6.4/pkgdb
  $HOME/.stack/programs/x86_64-osx/ghc-8.6.4/lib/ghc-8.6.4/package.conf.d
GHC_ENVIRONMENT=
  /private/var/folders/vt/k170lq_x09n8r_0n0pml2_8w0000gp/T/stack83997/test-ghc-env
HASKELL_PACKAGE_SANDBOX=
  $HOME/.stack/snapshots/x86_64-osx/2c51859fcfd0b0bbd239f96f11700837286b155df4d5d28072631deb79498000/8.6.4/pkgdb
HASKELL_PACKAGE_SANDBOXES=
  $HINT_ROOT/.stack-work/install/x86_64-osx/2c51859fcfd0b0bbd239f96f11700837286b155df4d5d28072631deb79498000/8.6.4/pkgdb
  $HOME/.stack/snapshots/x86_64-osx/2c51859fcfd0b0bbd239f96f11700837286b155df4d5d28072631deb79498000/8.6.4/pkgdb
HASKELL_DIST_DIR=
  $HINT_ROOT/.stack-work/dist/x86_64-osx/Cabal-2.4.0.1
DESTDIR (unset)
GHCRTS (unset)

gelisam · 2019-07-07T15:23:27Z

It was indeed GHC_ENVIRONMENT: calling unsetEnv "GHC_ENVIRONMENT" at the beginning of the test suite allows the tests to pass with stack 2.1.1 as well.

Before using this as a permanent fix, however, I would like to read more about GHC_ENVIRONMENT in order to understand what are the ramifications of unsetting it.

gelisam · 2019-07-07T15:41:24Z

Unsetting GHC_ENVIRONMENT caused ghc to use the .ghc.environment.x86_64-darwin-8.6.4 file which was created by an earlier cabal new-test run. But stack test still passes after removing the file, so the tests run fine with the environment created by cabal and in whatever the default environment is, but not in the environment created by stack!

gelisam · 2019-07-07T15:47:56Z

Aha! stack creates an environment which only includes the packages listed in the .cabal file, while cabal includes a bunch of extra packages (~~maybe all the boot packages?~~ no, it's all the recursive dependencies). stack's approach seems more principled, so I think the better fix is to add containers to the .cabal file.

ghc uses the GHC_ENVIRONMENT variable to find environment files describing which packages are visible. When running the tests using "cabal test", no environment file is created and GHC_ENVIRONMENT remains unset. ghc nevertheless has access to a number of default packages, including containers, so the test which imports Data.Map succeeds. When running the tests using "cabal v2-test", a local environment file is created, but GHC_ENVIRONMENT remains unset. ghc nevertheless finds the environment file, which includes all the dependencies listed in the .cabal file and also all their recursive dependencies, including containers, so the test which imports Data.Map succeeds. When running "stack test" using stack 1.9.3, no environment file is created and GHC_ENVIRONMENT remains unset. So, depending on whether a local environment file has been created by a previous run of "cabal v2-test", one of the two scenarios above occurs, and the test succeeds. When running "stack test" using stack 2.1.1, a temporary environment file is created in a temporary directory, and GHC_ENVIRONMENT points to it. That environment file inludes all the dependencies listed in the .cabal file, but _not_ their recursive dependencies. Since containers was not listed in the .cabal file, the test which imports Data.Map failed. This commit fixes the problem by adding containers to the .cabal file. See commercialhaskell/stackage#4668 for details.

gelisam · 2019-07-07T17:15:13Z

I have pushed a fix upstream, but I did not release a new version of hint. Since the fix only touches the .cabal file, I don't think we need to, it should be possible to fix it via a hackage revision instead.

@mvdan, could you grant me maintainer privileges for the hint package? https://hackage.haskell.org/package/hint/maintain tells me I don't have access.

mvdan · 2019-07-07T23:36:17Z

@gelisam sure, done.

gelisam · 2019-07-08T02:19:59Z

Thanks. Unfortunately, it looks like adding a new dependency is not a change which can be done as a hackage revision after all, so we'll have to release a new version. @mvdan, I ran the tests, I've bumped the version, I added an entry to the changelog, and I tagged the commit, is there anything else I need to do publishing a new version?

mvdan · 2019-07-08T11:17:53Z

@gelisam That's pretty much it. I also add the changelog info to GitHub's releases page, but that's up to you.

I've also been using semver (x.y.z without a fourth number) since I picked up the package, but I don't think that's terribly important :)

gelisam · 2019-07-08T11:27:18Z

@mihaimaruseac fixed upstream, please upgrade to hint-0.9.0.1.

The change is definitely backwards-compatible: all I did was to add containers to the list of dependencies of the test suite. The tests broke because stack 2.1.1 now uses the GHC_ENVIRONMENT variable to limit the packages which are visible at runtime to be the same as the packages which are visible at compilation-time.

mihaimaruseac · 2019-07-09T03:25:26Z

Awesome. Thank you @gelisam and sorry for being silent the past week. Real life took over

bergmark · 2019-07-09T10:02:24Z

Let's try it out.

mihaimaruseac added the failure: test-suite label Jun 25, 2019

mihaimaruseac added a commit that referenced this issue Jun 25, 2019

disable tests for #4667, #4668 and #4669

b3f1a7c

mihaimaruseac mentioned this issue Jun 26, 2019

Test failures on nightly on Stackage haskell-hint/hint#82

Closed

mihaimaruseac added a commit that referenced this issue Jun 28, 2019

Failing tests (pinging #4649, #4651, #4652, #4653, #4661, #4668, #4669,

3eee5c1

#4670)

gelisam mentioned this issue Jul 7, 2019

include the stack version in the Travis logs gelisam/hawk#210

Closed

gelisam mentioned this issue Jul 7, 2019

fix test suite with stack 2.x haskell-hint/hint#84

Merged

This was referenced Jul 8, 2019

haskell-gi test failure #4661

Closed

password test failures #4654

Closed

pretty-simple test failure #4652

Closed

servant-ruby test failure #4650

Closed

bergmark closed this as completed in a7b3d08 Jul 9, 2019

gelisam mentioned this issue Jun 16, 2020

Multiple embedded interpreter instances haskell-hint/hint#68

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

hint test failures #4668

hint test failures #4668

mihaimaruseac commented Jun 25, 2019

mvdan commented Jun 25, 2019

gelisam commented Jul 1, 2019 •

edited

Loading

gelisam commented Jul 1, 2019 •

edited

Loading

gelisam commented Jul 7, 2019

gelisam commented Jul 7, 2019 •

edited

Loading

gelisam commented Jul 7, 2019 •

edited

Loading

gelisam commented Jul 7, 2019

gelisam commented Jul 7, 2019 •

edited

Loading

gelisam commented Jul 7, 2019

gelisam commented Jul 7, 2019

gelisam commented Jul 7, 2019 •

edited

Loading

gelisam commented Jul 7, 2019 •

edited

Loading

mvdan commented Jul 7, 2019

gelisam commented Jul 8, 2019

mvdan commented Jul 8, 2019

gelisam commented Jul 8, 2019

mihaimaruseac commented Jul 9, 2019

bergmark commented Jul 9, 2019

hint test failures #4668

hint test failures #4668

Comments

mihaimaruseac commented Jun 25, 2019

mvdan commented Jun 25, 2019

gelisam commented Jul 1, 2019 • edited Loading

gelisam commented Jul 1, 2019 • edited Loading

gelisam commented Jul 7, 2019

gelisam commented Jul 7, 2019 • edited Loading

gelisam commented Jul 7, 2019 • edited Loading

gelisam commented Jul 7, 2019

gelisam commented Jul 7, 2019 • edited Loading

gelisam commented Jul 7, 2019

gelisam commented Jul 7, 2019

gelisam commented Jul 7, 2019 • edited Loading

gelisam commented Jul 7, 2019 • edited Loading

mvdan commented Jul 7, 2019

gelisam commented Jul 8, 2019

mvdan commented Jul 8, 2019

gelisam commented Jul 8, 2019

mihaimaruseac commented Jul 9, 2019

bergmark commented Jul 9, 2019

gelisam commented Jul 1, 2019 •

edited

Loading

gelisam commented Jul 1, 2019 •

edited

Loading

gelisam commented Jul 7, 2019 •

edited

Loading

gelisam commented Jul 7, 2019 •

edited

Loading

gelisam commented Jul 7, 2019 •

edited

Loading

gelisam commented Jul 7, 2019 •

edited

Loading

gelisam commented Jul 7, 2019 •

edited

Loading