Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

haskell builder: don't copy packages from GHCs #57706

Closed
wants to merge 3 commits into
base: master
from

Conversation

Projects
None yet
3 participants
@michaelpj
Copy link
Contributor

michaelpj commented Mar 15, 2019

In order to build the package databases that we will use when compiling
a Haskell package, we iterate over the relevant dependencies, and if
they contain a package db, we copy its contents over.

So far so good, except when one of those dependencies is GHC. This
doesn't happen ordinarily, but it will happen when we construct the
package database for compiling Setup.hs. This is compiled for the
build architecture, so we get the build deps, including both the native
and the cross GHC (if there is one).

In this case, we end up copying the packages from the GHC's package
database. This is at best unnecessary, since we will get those packages
from the GHC when we compile with it.

At worst, however, this is semantically questionable. We can end up
having multiple copies of e.g. Cabal with the same version, but
(potentially) different contents. At the moment, GHC will expose one of
these at semi-random depending on which one it looks at "first".
However, there is a MR open in GHC which as a
side effect will instead expose both, leading to ambiguous module
warnings (which is not unreasonable, since it is ambiguous).

So what can we do about it? The simplest solution is just to not copy
the package databases from GHC. GHC is special in this regard, so I
think it's okay to treat it specially.

This PR should have no effect on anything now, but will prevent any
breakage when/if the GHC patch lands.


I've opened this against haskell-updates - or should it be staging?

I've tested this by building our packages that use the nixpkgs Haskell infrastructure, using this patch and a GHC patched with the change in the linked MR. That builds and works fine on NixOS and macOS.

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

peti and others added some commits Mar 14, 2019

hackage-packages.nix: automatic Haskell package set update
This update was generated by hackage2nix v2.14.2 from Hackage revision
commercialhaskell/all-cabal-hashes@accf2e4.
hackage-packages.nix: automatic Haskell package set update
This update was generated by hackage2nix v2.14.2 from Hackage revision
commercialhaskell/all-cabal-hashes@cc9fd95.
haskell builder: don't copy packages from GHCs
In order to build the package databases that we will use when compiling
a Haskell package, we iterate over the relevant dependencies, and if
they contain a package db, we copy its contents over.

So far so good, except when one of those dependencies is GHC. This
doesn't happen ordinarily, but it will happen when we construct the
package database for compiling `Setup.hs`.  This is compiled for the
build architecture, so we get the build deps, including both the native
and the cross GHC (if there is one).

In this case, we end up copying the packages from the GHC's package
database. This is at best unnecessary, since we will get those packages
from the GHC when we compile with it.

At worst, however, this is semantically questionable. We can end up
having multiple copies of e.g. Cabal with the same version, but
(potentially) different contents. At the moment, GHC will expose one of
these at semi-random depending on which one it looks at "first".
However, there is a MR open [in
GHC](https://gitlab.haskell.org/ghc/ghc/merge_requests/545) which as a
side effect will instead expose both, leading to ambiguous module
warnings (which is not unreasonable, since it *is* ambiguous).

So what can we do about it? The simplest solution is just to not copy
the package databases from GHC. GHC is special in this regard, so I
think it's okay to treat it specially.

This PR should have no effect on anything now, but will prevent any
breakage when/if the GHC patch lands.

@michaelpj michaelpj requested review from basvandijk and peti as code owners Mar 15, 2019

@peti peti force-pushed the NixOS:haskell-updates branch 5 times, most recently from 2125549 to 6a6450b Mar 16, 2019

@peti peti changed the base branch from haskell-updates to master Mar 16, 2019

@peti

This comment has been minimized.

Copy link
Member

peti commented Mar 16, 2019

I use haskell-updates for release-19.03 at the moment. The branch should switch back to targeting master in a day or two, then I'll change pick up this change and merge it.

peti added a commit that referenced this pull request Mar 16, 2019

generic Haskell builder: don't copy packages from GHC
In order to build the package databases that we will use when compiling
a Haskell package, we iterate over the relevant dependencies, and if
they contain a package db, we copy its contents over.

So far so good, except when one of those dependencies is GHC. This
doesn't happen ordinarily, but it will happen when we construct the
package database for compiling `Setup.hs`.  This is compiled for the
build architecture, so we get the build deps, including both the native
and the cross GHC (if there is one).

In this case, we end up copying the packages from the GHC's package
database. This is at best unnecessary, since we will get those packages
from the GHC when we compile with it.

At worst, however, this is semantically questionable. We can end up
having multiple copies of e.g. Cabal with the same version, but
(potentially) different contents. At the moment, GHC will expose one of
these at semi-random depending on which one it looks at "first".
However, there is a MR open [in
GHC](https://gitlab.haskell.org/ghc/ghc/merge_requests/545) which as a
side effect will instead expose both, leading to ambiguous module
warnings (which is not unreasonable, since it *is* ambiguous).

So what can we do about it? The simplest solution is just to not copy
the package databases from GHC. GHC is special in this regard, so I
think it's okay to treat it specially.

This PR should have no effect on anything now, but will prevent any
breakage when/if the GHC patch lands.

Closes #57706.
@peti

This comment has been minimized.

Copy link
Member

peti commented Mar 16, 2019

Merged to haskell-updates in 19e0911. This will go to master once the test builds are complete.

@michaelpj

This comment has been minimized.

Copy link
Contributor Author

michaelpj commented Mar 16, 2019

Thanks!

peti added a commit that referenced this pull request Mar 16, 2019

generic Haskell builder: don't copy packages from GHC
In order to build the package databases that we will use when compiling
a Haskell package, we iterate over the relevant dependencies, and if
they contain a package db, we copy its contents over.

So far so good, except when one of those dependencies is GHC. This
doesn't happen ordinarily, but it will happen when we construct the
package database for compiling `Setup.hs`.  This is compiled for the
build architecture, so we get the build deps, including both the native
and the cross GHC (if there is one).

In this case, we end up copying the packages from the GHC's package
database. This is at best unnecessary, since we will get those packages
from the GHC when we compile with it.

At worst, however, this is semantically questionable. We can end up
having multiple copies of e.g. Cabal with the same version, but
(potentially) different contents. At the moment, GHC will expose one of
these at semi-random depending on which one it looks at "first".
However, there is a MR open [in
GHC](https://gitlab.haskell.org/ghc/ghc/merge_requests/545) which as a
side effect will instead expose both, leading to ambiguous module
warnings (which is not unreasonable, since it *is* ambiguous).

So what can we do about it? The simplest solution is just to not copy
the package databases from GHC. GHC is special in this regard, so I
think it's okay to treat it specially.

This PR should have no effect on anything now, but will prevent any
breakage when/if the GHC patch lands.

Closes #57706.

@peti peti closed this in 387c513 Mar 17, 2019

@michaelpj

This comment has been minimized.

Copy link
Contributor Author

michaelpj commented Mar 18, 2019

I'm not sure I quite followed the workflow process there, but I'd appreciate it if this got into release-19.03 as well if it hasn't already.

peti added a commit that referenced this pull request Mar 18, 2019

generic Haskell builder: don't copy packages from GHC
In order to build the package databases that we will use when compiling
a Haskell package, we iterate over the relevant dependencies, and if
they contain a package db, we copy its contents over.

So far so good, except when one of those dependencies is GHC. This
doesn't happen ordinarily, but it will happen when we construct the
package database for compiling `Setup.hs`.  This is compiled for the
build architecture, so we get the build deps, including both the native
and the cross GHC (if there is one).

In this case, we end up copying the packages from the GHC's package
database. This is at best unnecessary, since we will get those packages
from the GHC when we compile with it.

At worst, however, this is semantically questionable. We can end up
having multiple copies of e.g. Cabal with the same version, but
(potentially) different contents. At the moment, GHC will expose one of
these at semi-random depending on which one it looks at "first".
However, there is a MR open [in
GHC](https://gitlab.haskell.org/ghc/ghc/merge_requests/545) which as a
side effect will instead expose both, leading to ambiguous module
warnings (which is not unreasonable, since it *is* ambiguous).

So what can we do about it? The simplest solution is just to not copy
the package databases from GHC. GHC is special in this regard, so I
think it's okay to treat it specially.

This PR should have no effect on anything now, but will prevent any
breakage when/if the GHC patch lands.

Closes #57706.

(cherry picked from commit 387c513)

michaelpj added a commit to input-output-hk/nixpkgs that referenced this pull request Mar 18, 2019

generic Haskell builder: don't copy packages from GHC
In order to build the package databases that we will use when compiling
a Haskell package, we iterate over the relevant dependencies, and if
they contain a package db, we copy its contents over.

So far so good, except when one of those dependencies is GHC. This
doesn't happen ordinarily, but it will happen when we construct the
package database for compiling `Setup.hs`.  This is compiled for the
build architecture, so we get the build deps, including both the native
and the cross GHC (if there is one).

In this case, we end up copying the packages from the GHC's package
database. This is at best unnecessary, since we will get those packages
from the GHC when we compile with it.

At worst, however, this is semantically questionable. We can end up
having multiple copies of e.g. Cabal with the same version, but
(potentially) different contents. At the moment, GHC will expose one of
these at semi-random depending on which one it looks at "first".
However, there is a MR open [in
GHC](https://gitlab.haskell.org/ghc/ghc/merge_requests/545) which as a
side effect will instead expose both, leading to ambiguous module
warnings (which is not unreasonable, since it *is* ambiguous).

So what can we do about it? The simplest solution is just to not copy
the package databases from GHC. GHC is special in this regard, so I
think it's okay to treat it specially.

This PR should have no effect on anything now, but will prevent any
breakage when/if the GHC patch lands.

Closes NixOS#57706.

angerman added a commit to input-output-hk/nixpkgs that referenced this pull request Mar 18, 2019

generic Haskell builder: don't copy packages from GHC
In order to build the package databases that we will use when compiling
a Haskell package, we iterate over the relevant dependencies, and if
they contain a package db, we copy its contents over.

So far so good, except when one of those dependencies is GHC. This
doesn't happen ordinarily, but it will happen when we construct the
package database for compiling `Setup.hs`.  This is compiled for the
build architecture, so we get the build deps, including both the native
and the cross GHC (if there is one).

In this case, we end up copying the packages from the GHC's package
database. This is at best unnecessary, since we will get those packages
from the GHC when we compile with it.

At worst, however, this is semantically questionable. We can end up
having multiple copies of e.g. Cabal with the same version, but
(potentially) different contents. At the moment, GHC will expose one of
these at semi-random depending on which one it looks at "first".
However, there is a MR open [in
GHC](https://gitlab.haskell.org/ghc/ghc/merge_requests/545) which as a
side effect will instead expose both, leading to ambiguous module
warnings (which is not unreasonable, since it *is* ambiguous).

So what can we do about it? The simplest solution is just to not copy
the package databases from GHC. GHC is special in this regard, so I
think it's okay to treat it specially.

This PR should have no effect on anything now, but will prevent any
breakage when/if the GHC patch lands.

Closes NixOS#57706.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.