Reexported modules, take 2 #385
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Cabal's `reexported-modules` field allows one to reexport and optionally
rename modules from a library. This commit adds an `exports` attribute
to `haskell_library` which accomplishes the same, accepting a dictionary
mapping dependencies to strings using the same syntax. For example:
```
haskell_library(
...,
exports = {
":dependency": "A, B as BNew",
},
...
)
```
Registration of a new package consists in, 1. copying the registration file into the package db, 2. performing some validation on the registration file content, 3. recaching, i.e. regenerating the package db cache file. Normally, this is all done by `ghc-pkg register`. But in our case, `ghc-pkg register` is painful, because the validation it performs is slow, somewhat redundant but especially, too strict (see e.g. https://ghc.haskell.org/trac/ghc/ticket/15478). So we now do (1) and (3) manually, by copying then calling `ghc-pkg recache` directly. The downside is that we do lose the few validations that `ghc-pkg register` was doing that was useful. e.g. when reexporting modules, validation checks that the source module does exist. We'll want to come back to using `ghc-pkg register` eventually, but for now https://ghc.haskell.org/trac/ghc/ticket/15478 is a blocker.
mboes
force-pushed
the
reexported-modules
branch
from
69f2dd9
to
aa35548
Compare
mrkkrp
reviewed
Aug 6, 2018
|
Yes, the exported-modules list needs some post processing now to make it cromulent to the REPL. |
|
Looks good to me. I still don't think it'd be the end of the world to have |
mboes
added a commit
that referenced
this pull request
Aug 9, 2018
In #385, we claimed to exploit the fact that GHC only needs package names in reexport declarations to obviate the need for knowing the versions (and dependency hashes) of all prebuilt packages. We provided a test to substantiate this indeed works. But it turns that the test was not a good test. The test was flawed because it featured libraries reexporting `base`. It turns out that `base` (along with `rts`) are magical in GHC. Because they are so-called "wired-in packages", they are assumed unique and are therefore not located using a version number. So while the scheme in #385 worked for `base`, it didn't work in the general case, as observed by @lunaris. The fix consists qualifying all package reexports with full package id's, not package names. How do we derive a package id given only a package name? The solution is to have the toolchain generate a dump of the global package database that ships with the compiler. This is only needed once for the entire workspace. From this dump, we can lookup the full package id given any particular name. Fixes #357.
mboes
added a commit
that referenced
this pull request
Aug 10, 2018
In #385, we claimed to exploit the fact that GHC only needs package names in reexport declarations to obviate the need for knowing the versions (and dependency hashes) of all prebuilt packages. We provided a test to substantiate this indeed works. But it turns that the test was not a good test. The test was flawed because it featured libraries reexporting `base`. It turns out that `base` (along with `rts`) are magical in GHC. Because they are so-called "wired-in packages", they are assumed unique and are therefore not located using a version number. So while the scheme in #385 worked for `base`, it didn't work in the general case, as observed by @lunaris. The fix consists qualifying all package reexports with full package id's, not package names. How do we derive a package id given only a package name? The solution is to have the toolchain generate a dump of the global package database that ships with the compiler. This is only needed once for the entire workspace. From this dump, we can lookup the full package id given any particular name. Fixes #357.
mboes
added a commit
that referenced
this pull request
Aug 10, 2018
In #385, we claimed to exploit the fact that GHC only needs package names in reexport declarations to obviate the need for knowing the versions (and dependency hashes) of all prebuilt packages. We provided a test to substantiate this indeed works. But it turns that the test was not a good test. The test was flawed because it featured libraries reexporting `base`. It turns out that `base` (along with `rts`) are magical in GHC. Because they are so-called "wired-in packages", they are assumed unique and are therefore not located using a version number. So while the scheme in #385 worked for `base`, it didn't work in the general case, as observed by @lunaris. The fix consists qualifying all package reexports with full package id's, not package names. How do we derive a package id given only a package name? The solution is to have the toolchain generate a dump of the global package database that ships with the compiler. This is only needed once for the entire workspace. From this dump, we can lookup the full package id given any particular name. Fixes #357.
mboes
added a commit
that referenced
this pull request
Aug 10, 2018
In #385, we claimed to exploit the fact that GHC only needs package names in reexport declarations to obviate the need for knowing the versions (and dependency hashes) of all prebuilt packages. We provided a test to substantiate this indeed works. But it turns that the test was not a good test. The test was flawed because it featured libraries reexporting `base`. It turns out that `base` (along with `rts`) are magical in GHC. Because they are so-called "wired-in packages", they are assumed unique and are therefore not located using a version number. So while the scheme in #385 worked for `base`, it didn't work in the general case, as observed by @lunaris. The fix consists qualifying all package reexports with full package id's, not package names. How do we derive a package id given only a package name? The solution is to have the toolchain generate a dump of the global package database that ships with the compiler. This is only needed once for the entire workspace. From this dump, we can lookup the full package id given any particular name. Fixes #357.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is effectively a reopen of #381. Starting from the same code,
I made three changes:
master,a pkg_id like a haskell_library does,
ghc-pkg registertoghc-pkg cache.The key observation that made this PR possible is that while
ghc-pkg registeris very picky about how we name packages e.g. independsand
exposed-modulesfields, GHC itself is much less so. GHC is quitehappy for us to not specify a version number, let alone a package
hash. It's probably a bug that
ghc-pkg registeris this picky. Seehttps://ghc.haskell.org/trac/ghc/ticket/15478.
We side step this bug by simply not using
ghc-pkg register. This isa temporary workaround, because that command does some useful sanity
checks we're not getting the benefit of anymore.