Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ghc envs not properly ignored by new-build #4010

Closed
hvr opened this issue Oct 19, 2016 · 8 comments · Fixed by #4433
Closed

ghc envs not properly ignored by new-build #4010

hvr opened this issue Oct 19, 2016 · 8 comments · Fixed by #4433

Comments

@hvr
Copy link
Member

hvr commented Oct 19, 2016

I just ran into an issue which with the help of @ezyang was diagnosed to be caused
by an unlucky combination of custom Setup.hs and a GHC 8.0.2 snapshot with
a not yet updated Cabal lib

The symptom is the following:

given a trivial package such as

name:                dummy
version:             0
build-type:          Simple
cabal-version:       >=1.10

library
  build-depends: tasty

Running cabal new-build for the first time succeeds, and creates a file .ghc.environment.x86_64-linux-8.0.1.20161014 with the content

-- This is a GHC environment file written by cabal. This means you can
-- run ghc or ghci and get the environment of the project as a whole.
-- But you still need to use cabal repl $target to get the environment
-- of specific components (libs, exes, tests etc) because each one can
-- have its own source dirs, cpp flags etc.
-- 
clear-package-db
global-package-db
package-db /home/hvr/.cabal/store/ghc-8.0.1.20161014/package.db
package-db dist-newstyle/packagedb/ghc-8.0.1.20161014
package-id ansi-terminal-0.6.2.3-e8175c6cd482d5c9e7b92ba68d6363b841b6627942fc6368b7980e5944277f9d
package-id base-4.9.0.0
package-id ghc-prim-0.5.0.0
package-id rts
package-id integer-gmp-1.0.0.1
package-id unix-2.7.2.0
package-id bytestring-0.10.8.1
package-id deepseq-1.4.2.0
package-id array-0.5.1.1
package-id time-1.6.0.1
package-id async-2.1.0-65beb88cd84537d902b2bb0f834cec19c2fc7a501af9de365360f8d1d3224d6a
package-id stm-2.4.4.1-4fb640c22a961fa87e0f7ca5aaa4a3d040a98d232ed1827dd3d2c534ddf5a059
package-id clock-0.7.2-e14c7ff6e51334938f8f342093fa84a10e8a685303527380a41ac7c5b9b8b278
package-id containers-0.5.7.1
package-id mtl-2.2.1-09ef0d4c975ddca7091852fddcdf250bc2243b8c25c9e267e3c43a1a2aeb8f14
package-id transformers-0.5.2.0
package-id optparse-applicative-0.13.0.0-fa1c47887f70127fe79d25c05562b407fe6f73cba4ff63d861ded51d43251a61
package-id ansi-wl-pprint-0.6.7.3-36d4937e3d5870272653a83aaa85f2b99285ab119ddcbde6b50d4f726cbab73a
package-id process-1.4.2.0
package-id directory-1.2.6.2
package-id filepath-1.4.1.0
package-id transformers-compat-0.5.1.4-795f7daed226c475a0f42d64d6a997ab25d35660d2882080b8c3d27ea749cfeb
package-id text-1.2.2.1-7d47767d7e9da8b3b49e4a3e24e417492dede5a9823bcec3284cc8b02d22c04e
package-id binary-0.8.3.0
package-id regex-base-0.93.2-e578239124026e62ea947d16d81304c8ef35f86d34d23706baae8b5d9282929f
package-id tagged-0.8.5-7c78a1c83a8b86913c62b59d223ee6d945f5b40f6dd9a65bedaa0c736b1cfe02
package-id template-haskell-2.11.0.0
package-id ghc-boot-th-8.0.1.20161014
package-id pretty-1.1.3.3

However, after resetting the store via e.g. rm -rf ~/.cabal/store/ghc-8.0.1.20161014/ and removing dist-newstyle (but leaving the .ghc.env* file as-is), and re-running cabal new-build we now run into a weird & confusing build failure:

$ cabal new-build
In order, the following will be built (use -v for more details):
 - ansi-terminal-0.6.2.3 (lib) (requires build)
 - clock-0.7.2 (lib) (requires build)
 - mtl-2.2.1 (lib) (requires build)
 - old-locale-1.0.0.7 (lib) (requires build)
 - stm-2.4.4.1 (lib) (requires build)
 - text-1.2.2.1 (lib) (requires build)
 - transformers-compat-0.5.1.4 (lib) (requires build)
 - ansi-wl-pprint-0.6.7.3 (lib) (requires build)
 - regex-base-0.93.2 (lib) (requires build)
 - old-time-1.1.0.3 (lib) (requires build)
 - async-2.1.0 (lib) (requires build)
 - parsec-3.1.11 (lib) (requires build)
 - tagged-0.8.5 (lib) (requires build)
 - optparse-applicative-0.13.0.0 (lib) (requires build)
 - unbounded-delays-0.1.0.9 (lib:unbounded-delays) (requires build)
 - regex-tdfa-1.2.2 (lib) (requires build)
 - tasty-0.11.0.4 (lib) (requires build)
 - dummy-0 (lib) (first run)
Configuring ansi-terminal-0.6.2.3 (lib)...
Configuring clock-0.7.2 (lib)...
Configuring mtl-2.2.1 (lib)...
Configuring old-locale-1.0.0.7 (lib)...
Configuring transformers-compat-0.5.1.4 (lib)...
Configuring stm-2.4.4.1 (lib)...
Configuring text-1.2.2.1 (lib)...
Building clock-0.7.2 (lib)...
Building mtl-2.2.1 (lib)...
Building ansi-terminal-0.6.2.3 (lib)...
Building old-locale-1.0.0.7 (lib)...
Building stm-2.4.4.1 (lib)...
Building text-1.2.2.1 (lib)...
Building transformers-compat-0.5.1.4 (lib)...
Configuring tagged-0.8.5 (lib)...
Configuring old-time-1.1.0.3 (lib)...
Building tagged-0.8.5 (lib)...
Configuring async-2.1.0 (lib)...
Building async-2.1.0 (lib)...
Building old-time-1.1.0.3 (lib)...
Configuring regex-base-0.93.2 (lib)...
Building regex-base-0.93.2 (lib)...
Configuring ansi-wl-pprint-0.6.7.3 (lib)...
Building ansi-wl-pprint-0.6.7.3 (lib)...
Configuring optparse-applicative-0.13.0.0 (lib)...
Building optparse-applicative-0.13.0.0 (lib)...
Configuring unbounded-delays-0.1.0.9 (all, due to Custom setup)...

Failed to build unbounded-delays-0.1.0.9. The failure occurred during the
configure step.
Build log (
/home/hvr/.cabal/logs/ghc-8.0.1.20161014/unbounded-delays-0.1.0.9-3018361f6c155727e30a06ab7e7c36258ec4708e839db0dd5e568528427ca8b8.log
):
[1 of 1] Compiling Main             ( /home/hvr/Haskell/dummy/dist-newstyle/tmp/build-24031/setup/setup.hs, /home/hvr/Haskell/dummy/dist-newstyle/tmp/build-24031/setup/Main.o )
Linking /home/hvr/Haskell/dummy/dist-newstyle/tmp/build-24031/setup/setup ...
Configuring unbounded-delays-0.1.0.9...
<command line>: cannot satisfy -package-id optparse-applicative-0.13.0.0-fa1c47887f70127fe79d25c05562b407fe6f73cba4ff63d861ded51d43251a61
    (use -v for more information)
cabal-head: Failed to build unbounded-delays-0.1.0.9 (which is required by
dummy). See the build log above for details.
@hvr hvr added the type: bug label Oct 19, 2016
@ezyang
Copy link
Contributor

ezyang commented Oct 19, 2016

CC @dcoutts who is captain of this feature.

@ezyang ezyang added this to the 2.0 milestone Oct 19, 2016
@dcoutts
Copy link
Contributor

dcoutts commented Oct 19, 2016

Sigh. I guess we can probably work around it for older Cabal Setup.hs scripts by passing --ghc-options=-hide-all-packages. This will also be helped by the things being back-ported to the 1.24.x branch.

@ezyang
Copy link
Contributor

ezyang commented Oct 20, 2016

So, this is because GHC walks up directories until it finds the environment file? Harsh. (You would quite prefer it if you could run Setup script with the environment file unconditionally hidden.) Maybe we could add some sort of environment variable escape hatch to GHC?

@bgamari
Copy link
Contributor

bgamari commented Nov 21, 2016

I wonder whether @ezyang is right; using -hide-all-packages to disable environment file support seems diabolically roundabout. If we are going to do this it should be very soon; the window for 8.0.2 is very rapidly closing.

@dcoutts
Copy link
Contributor

dcoutts commented Nov 22, 2016

@bgamari it could be more explicit with its own new flag, and perhaps that's nice, but it does not seem essential, and certainly not something to hold up 8.0.2 for. The -hide-all-packages while not very explicit does have the merit that it is what is done in 99% of cases already when setting up a specific environment.

ezyang added a commit to ezyang/cabal that referenced this issue Nov 25, 2016
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>
ezyang added a commit that referenced this issue Nov 26, 2016
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>
@alexbiehl
Copy link
Member

alexbiehl commented Jan 15, 2017

I really like this feature and would want it to get into GHC as soon as possible. I have created a patch for GHC: alexbiehl/ghc@42bbc38

If this approach is okay I will submit this patch to GHCHQ and modify cabal accordingly.

alexbiehl added a commit to alexbiehl/cabal that referenced this issue Jan 15, 2017
Hopefully fixes haskell#4010.

Parallel to this commit there will be a differential for a new command
line flag in GHC: `-no-implicit-package-env`. This flag explicitly
disables the implicit search for package.env files in GHC.

Before this patch GHC abused the `-hide-all-packages` flag to disable
package.env search which lead to the behaviour in haskell#4010.
@ezyang
Copy link
Contributor

ezyang commented Jan 16, 2017

Hi @alexbiehl, the GHC-level change doesn't really solve the core problem with this bug, because the issue here is old Cabal Setup scripts aren't passing enough flags to environment file enabled GHCs to ensure that the environment file is ignored on certain operations.

So really, your fix "accidentally" solves the problem, but only because we never write out environment variables for GHC 8.0 which ships with Cabal 1.24 which doesn't pass -hide-all-packages in enough places. Maybe that's good enough, but I am still concerned that there is no consistent strategy in Cabal's codebase for making sure we don't forget to pass -hide-all-packages to GHC to ensure that the package environment is not applied.

@23Skidoo 23Skidoo modified the milestones: 2.0.1, 2.0 Feb 17, 2017
@hvr
Copy link
Member Author

hvr commented Mar 31, 2017

There appear to be at least two ways to workaround this:

  • either setenv("GHC_ENVIRONMENT","/dev/null") (or rather set env-var via execve(2)) before executing a Setup.hs linked against a Cabal < 1.24.2 (NB: this results in an implicit -hide-all-packages), or
  • inject --ghc-options=-hide-all-packages into configPrograms (in ProjectPlanning.setupHsConfigureFlags), e.g.
    configProgramArgs
        | elabSetupScriptCliVersion < mkVersion [1,24,2]
          -- workaround for https://github.com/haskell/cabal/issues/4010
                              = Map.toList $ traceShowId $
                                Map.insertWith (++) "ghc" ["-hide-all-packages"]
                                               elabProgramArgs
        | otherwise           = Map.toList elabProgramArgs

Note however, that injecting --ghc-options=-hide-all-packages appears to result in duplication of the -hide-all-packages flag:

("/opt/ghc/bin/ghc",["--make","-v","-fbuilding-cabal-package","-O","-static","-dynamic-too","-dynosuf","dyn_o","-dynhisuf","dyn_hi","-outputdir","dist/build","-odir","dist/build","-hidir","dist/build","-stubdir","dist/build","-i","-idist/build","-i.","-idist/build/autogen","-Idist/build/autogen","-Idist/build","-optP-include","-optPdist/build/autogen/cabal_macros.h","-this-unit-id","unbounded-delays-0.1.0.10-a6e8369d7f8cef5fd41d8a3108ced8cc60762d154f538655988622577ad44308"

,"-hide-all-packages","-no-user-package-db","-package-db","/home/test/.cabal/store/ghc-8.0.2/package.db","-package-db","dist/package.conf.inplace","-package-id","base-4.9.1.0","-XHaskell98","Control.Concurrent.Thread.Delay","Control.Concurrent.Timeout","-Wall"

,"-hide-all-packages"])

PS: For a proof of concept, the shell invocation GHC_ENVIRONMENT=/dev/null cabal new-build can be used to test the workaround

hvr added a commit to hvr/cabal that referenced this issue Apr 2, 2017
This was temporarily disabled via
3033776 due to haskell#4010
but it turns out that we can easily workaround this for
older Cabal versions.

All we need to do is inject `--ghc-options=-hide-all-packages` into
the flags passed to Setup.hs when an old `Cabal` version is detected
(this was suggested by @dcoutts in
haskell#4010 (comment))

Luckily, `-hide-all-packages` is idempotent in GHC, so we can place it
anywhere on the GHC command-line as well as multiple times with the
same result.
hvr added a commit to hvr/cabal that referenced this issue Apr 2, 2017
This was temporarily disabled via
3033776 due to haskell#4010
but it turns out that we can easily workaround this for
older Cabal versions.

All we need to do is inject `--ghc-options=-hide-all-packages` into
the flags passed to Setup.hs when an old `Cabal` version is detected
(this was suggested by @dcoutts in
haskell#4010 (comment))

Luckily, `-hide-all-packages` is idempotent in GHC, so we can place it
anywhere on the GHC command-line as well as multiple times with the
same result.

This would close haskell#4010
hvr added a commit to hvr/cabal that referenced this issue Apr 2, 2017
This was temporarily disabled via
3033776 due to haskell#4010
but it turns out that we can easily workaround this for
older Cabal versions.

All we need to do is inject `--ghc-options=-hide-all-packages` into
the flags passed to Setup.hs when an old `Cabal` version is detected
(this was suggested by @dcoutts in
haskell#4010 (comment))

Luckily, `-hide-all-packages` is idempotent in GHC, so we can place it
anywhere on the GHC command-line as well as multiple times with the
same result.

This would close haskell#4010
hvr added a commit to hvr/cabal that referenced this issue Apr 2, 2017
This was temporarily disabled via
3033776 due to haskell#4010
but it turns out that we can easily workaround this for
older Cabal versions.

All we need to do is inject `--ghc-options=-hide-all-packages` into
the flags passed to Setup.hs when an old `Cabal` version is detected
(this was suggested by @dcoutts in
haskell#4010 (comment))

Luckily, `-hide-all-packages` is idempotent in GHC, so we can place it
anywhere on the GHC command-line as well as multiple times with the
same result.

This would close haskell#4010
hvr added a commit to hvr/cabal that referenced this issue May 15, 2017
This was temporarily disabled via
3033776 due to haskell#4010
but it turns out that we can easily workaround this for
older Cabal versions.

All we need to do is inject `--ghc-options=-hide-all-packages` into
the flags passed to Setup.hs
(this was suggested by @dcoutts in
haskell#4010 (comment))

Luckily, `-hide-all-packages` is idempotent in GHC, so we can place it
anywhere on the GHC command-line as well as multiple times with the
same result.

This would close haskell#4010
@hvr hvr closed this as completed in #4433 May 15, 2017
bgamari pushed a commit to bgamari/cabal that referenced this issue Jul 29, 2017
This was temporarily disabled via
3033776 due to haskell#4010
but it turns out that we can easily workaround this for
older Cabal versions.

All we need to do is inject `--ghc-options=-hide-all-packages` into
the flags passed to Setup.hs when an old `Cabal` version is detected
(this was suggested by @dcoutts in
haskell#4010 (comment))

Luckily, `-hide-all-packages` is idempotent in GHC, so we can place it
anywhere on the GHC command-line as well as multiple times with the
same result.

This would close haskell#4010
bgamari pushed a commit to bgamari/cabal that referenced this issue Aug 28, 2017
This was temporarily disabled via
3033776 due to haskell#4010
but it turns out that we can easily workaround this for
older Cabal versions.

All we need to do is inject `--ghc-options=-hide-all-packages` into
the flags passed to Setup.hs when an old `Cabal` version is detected
(this was suggested by @dcoutts in
haskell#4010 (comment))

Luckily, `-hide-all-packages` is idempotent in GHC, so we can place it
anywhere on the GHC command-line as well as multiple times with the
same result.

This would close haskell#4010

(cherry picked from commit 6fc90eb34259170aa4949d2d127c942899f3351e)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants