Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to define template Haskell dependencies separately #1382

Open
aherrmann opened this issue Jul 1, 2020 · 2 comments
Open

Allow to define template Haskell dependencies separately #1382

aherrmann opened this issue Jul 1, 2020 · 2 comments
Labels
P3 minor: not priorized type: feature request

Comments

@aherrmann
Copy link
Member

aherrmann commented Jul 1, 2020

Is your feature request related to a problem? Please describe.
Right now Haskell compilation for haskell_library|binary|test, i.e. haskell/private/actions.compile.bzl, requires interface files, object files, and linked libraries of dependencies, as well as direct and transitive C libraries as inputs. The reason is that the target may contain template Haskell code in which case GHC needs to load some of these linked libraries at compile time.

However, in many cases the dependency on linked libraries is not necessary and disables potential build parallelism as well as cache hits if interface files did not change but linked libraries did.

Additionally, the fact that there is no distinction between regular dependencies and template Haskell dependencies prevents us from making runtime dependencies (data attribute) of template Haskell dependencies available during compilation. See #1337 (comment).

Describe the solution you'd like
Add a th_deps attribute to haskell_library|binary|test that allows to distinguish regular and template Haskell dependencies as described above.

Describe alternatives you've considered
In the specific case of data dependencies of template Haskell the issue can be worked around with extra_srcs and CPP as described here.

@patrickt
Copy link

patrickt commented Sep 8, 2020

N.B. it would be great if this could apply both to haskell_repl and haskell_library targets.

@ylecornec ylecornec self-assigned this Apr 9, 2021
@facundominguez facundominguez changed the title Allow to defined template Haskell dependencies separately Allow to define template Haskell dependencies separately Aug 26, 2021
jcpetruzza added a commit to jcpetruzza/rules_haskell that referenced this issue Feb 9, 2022
Implements the proposal in tweag#1382. A new `th_deps` attribute
is added to `haskell_(library|binary|test)` that is similar
to `deps` but makes only the libraries in the formaer available
at compile-time. Dependencies in `th_deps` are also made
available during linking: FFI declarations also need to be
resolved at link time, even if they were used only for the
sake of TH.

As discussed in the issue, this reduces the number of
dependencies for the build action, creating more
opportunities for parallel work.

In addition, adding all cc deps at compile time was
preventing some configurations from working: ghc's dynamic linker
apparently tries to load the .so files in the order in which they
are specified so if a.so declares symbols referenced by b.so,
a.so needs to be given first. We don't seem to control the order
in which these are passed from the rules, though. E.g., without
`th_deps` defaulting to `[]` I'm unable to package llvm-hs
to use a bazel-built llvm, since the former uses TH and FFI
(but no FFI during TH) and the latter splits things in several
several packages.
@ylecornec ylecornec removed their assignment Feb 9, 2022
jcpetruzza added a commit to jcpetruzza/rules_haskell that referenced this issue Feb 14, 2022
Implements the proposal in tweag#1382. A new `th_deps` attribute
is added to `haskell_(library|binary|test)` that is similar
to `deps` but makes only the libraries in the formaer available
at compile-time.

As discussed in the issue, this reduces the number of
dependencies for the build action, creating more
opportunities for parallel work.

In addition, adding all cc deps at compile time was
preventing some configurations from working: ghc's dynamic linker
apparently tries to load the .so files in the order in which they
are specified so if a.so declares symbols referenced by b.so,
a.so needs to be given first. We don't seem to control the order
in which these are passed from the rules, though. E.g., without
`th_deps` defaulting to `[]` I'm unable to package llvm-hs
to use a bazel-built llvm, since the former uses TH and FFI
(but no FFI during TH) and the latter splits things in several
several packages.
jcpetruzza added a commit to jcpetruzza/rules_haskell that referenced this issue Feb 14, 2022
Implements the proposal in tweag#1382. A new `th_deps` attribute
is added to `haskell_(library|binary|test)` that is similar
to `deps` but makes only the libraries in the formaer available
at compile-time.

As discussed in the issue, this reduces the number of
dependencies for the build action, creating more
opportunities for parallel work.

In addition, adding all cc deps at compile time was
preventing some configurations from working: ghc's dynamic linker
apparently tries to load the .so files in the order in which they
are specified so if a.so declares symbols referenced by b.so,
a.so needs to be given first. We don't seem to control the order
in which these are passed from the rules, though. E.g., without
`th_deps` defaulting to `[]` I'm unable to package llvm-hs
to use a bazel-built llvm, since the former uses TH and FFI
(but no FFI during TH) and the latter splits things in several
several packages.
@aherrmann
Copy link
Member Author

aherrmann commented Feb 15, 2022

This is related to #878.

Describe alternatives you've considered
Ideally we would list Template Haskell dependencies separately, in a new attribute called compile_only_deps or similar.

@Xophmeister Xophmeister added the P3 minor: not priorized label Mar 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P3 minor: not priorized type: feature request
Projects
None yet
Development

No branches or pull requests

4 participants