cabal-install doesn't consider build-tools as dependencies #220

Open
bos opened this Issue May 24, 2012 · 14 comments
@bos
Haskell member
bos commented May 24, 2012 edited

Summary by @ezyang. When you write build-tools: alex, Cabal only checks that an executable of this name is installed; cabal-install does not know that if there is no such executable, alex needs to be installed. One complication is that build-tools does not necessarily correspond to a package, so it is not a priori obvious to cabal-install what should be installed to get an executable.

Here is my proposal:

  • We introduce a new type of dependency, a build-tools dependency, similar to how setup dependencies work.
  • We modify the solver to interpret each field of build-tools as a package name, and we solve for each build-tool as a separate build-tools dependency (so we don't do something goofy like try to solve for happy and alex's dependency sets consistently when they really couldn't care less.)
  • Some build-tools fields don't correspond to actual packages. We give two escape hooks in this situation. First, Cabal has an internal mapping of build tools to Maybe PackageName. If Nothing, this indicates that the build-tool is not managed by Cabal and we shouldn't try to solve for it. If Just pn, then it means that installing pn will also install the requested executable. This mapping will be configurable using the build-tool-packages field in a cabal-project and also modifiable by command line argument. (Names can be bikeshedded of course.) Our second escape hook is, if a build-tool refers to an unknown package name, we print a warning but continue to solve, simply ignoring that package. A user can suppress the warning by disabling this using the aforementioned flag.

(Imported from Trac #227, reported by guest on 2008-02-01)

On #haskell, nelhage mentioned that one of the bugs he ran into while trying to install Yi-0.3 through cabal-install was an error which looked like this:

Installing: /home/gwern/.cabal/lib/regex-posix-0.72.0.2/ghc-6.8.2/regex-posix-0.72.0.2/ghc-6.8.2
Registering regex-posix-0.72.0.2...
Reading package info from "dist/installed-pkg-config" ... done.
Saving old package config file... done.
Writing new package config file... done.
'yi-0.3' is cached.
[1 of 1] Compiling Main             ( Setup.hs, dist/setup/Main.o )
Linking dist/setup/setup ...
Configuring yi-0.3...
setup: alex version >=2.0.1&&<3 is required but it could not be found.

That is, the dependency on Alex (a version which satisfied >=2.0.1&&<3) was not met. Perfectly sensible as he did not have Alex installed. Puzzled, he did a 'cabal install alex', which worked, and then re-started. (I guess he had his GHC installed locally?).

He and I were puzzled, because cabal-install is supposed/usually does track down dependencies. Is build-tools excluded for a reason, or was it just accidentally omitted from dependency-tracking? If the latter, I think this is a bit of a bug.

(I use Cabal-1.2.3.0, btw, and the error message above was my reproduction of Nelhage's problem. I had to uninstall Alex from my system, but that did the trick, and reinstalling bypassed the issue just fine - which is why I hadn't seen it before.)

gwern

@dcoutts dcoutts was assigned May 24, 2012
@bos
Haskell member

(Imported comment by @dcoutts on 2008-02-01)

One problem is that not all build-tools correspond to haskell packages. Some do some don't. We have a hard coded list of them at the moment (which can be extended in Setup.hs files) so we could extend that with what haskell package if any the tools correspond to. Any better suggestions to make it a tad more generic?

@bos
Haskell member

(Imported comment by @dcoutts on 2008-02-28)

This limitation clearly confuses people.

  • [http://www.haskell.org/pipermail/cabal-devel/2008-June/003220.html](http://www.haskell.org/pipermail/cabal-devel/2008-June/003220.html)
  • [http://www.haskell.org/pipermail/haskell-cafe/2008-June/044222.html](http://www.haskell.org/pipermail/haskell-cafe/2008-June/044222.html)
@bos
Haskell member

(Imported comment by darrint on 2008-06-16)

Seems like what you need is to be able to determine which packages provide which executables. Alex, for instance, declares that it provides an executable by that name. For the next release of cabal-install, maybe change the package repository metadata format to include more (all?) the cabal info instead of a file list. That will allow you to solve more situations and do a full install plan up front.

Would that work?

@bos
Haskell member

(Imported comment by @dcoutts on 2008-06-16)

See also #342

@bos
Haskell member

(Imported comment by @mcandre on 2009-01-05)

I have the same problem with ncurses and c2hs. I'd really like dependencies to be handled automatically whenever possible.

@mietek

Looking forward to this being fixed.

@tibbe
Haskell member

@mietek right now this is blocked on no one having the time to work on this. If you're interested perhaps you could take a look? If you have any questions you can write to cabal-devel@haskell.org.

@mietek

@tibbe: I can't right now, but I am interested in solving this, in one way or another.

@mietek

Halcyon supports declaring and automatically installing build-tools and other Haskell apps for use at build-time with the HALCYON_SANDBOX_EXTRA_APPS option. Version constraints for these apps can be declared with the HALCYON_SANDBOX_EXTRA_APPS_CONSTRAINTS option.

Similarly, you can use HALCYON_EXTRA_APPS and HALCYON_EXTRA_APPS_CONSTRAINTS to declare additional apps to be installed for use at run-time.

See Haskell Language for an example of declaring alex and happy as sandbox extra apps.

@danse

This is quite a big flaw. I am an Haskell newbie, but i will try to put my two cents in order to keep the discussion alive

One problem is that not all build-tools correspond to haskell packages. Some do some don't. We have a hard coded list of them at the moment (which can be extended in Setup.hs files) so we could extend that with what haskell package if any the tools correspond to. Any better suggestions to make it a tad more generic?

What is the purpose of build-tools if not to list items to be installed? Correct me if i am wrong: if a build tool is declared as a build dependency in build-depends, it will be correctly installed, but if it is listed within build-tools it might be ignored?

@oneeyedziggy

"Every little bit helps," said the gnat and it pissed in the sea.
here's my little bit... I'm also a first timer like danse above (who i suspect is going to OsCon 2015), this kind of puts a sour taste in the mouth of anyone trying to pick up haskell...

@hvr
Haskell member

IMHO to keep this KISS-y, build-tools should be taken into account when solving for an install-plan, and try to find an install-plan that is possible with the currently available tools. I.e. consider build-tools a static (non-upgradeable/installable) property of the build-environment (like e.g. the GHC version -- which ultimately is a build-tool as well).

If, at some point in the future, we want to support auto-installing build-tools, we'd have to massively extend cabal-installs support for tracking package executables.

/cc @kosmikus

@mietek

@hvr’s suggestion would certainly make the experience less frustrating.

As it is, you’ll only find out you’re missing a build-tool halfway through building your dependencies — and that’s if you’re paying attention, because #1924 means unless you manually abort the build, you’ll continue burning CPU cycles on something which has no chance of succeeding.

@gasi gasi added a commit to gasi/SublimeHaskell that referenced this issue Jan 4, 2016
@gasi gasi Document `hsdev` issues without `happy`
Fixes #184
Fixes #240

See:
- b2a391257ccff8e191fde98ba949e91fa65e6daf
- haskell/cabal#220
ec9c0b2
@gasi gasi referenced this issue in SublimeHaskell/SublimeHaskell Jan 4, 2016
Closed

Document `hsdev` issues without `happy` #243

@ezyang ezyang added a commit to ezyang/cabal that referenced this issue Mar 31, 2016
@ezyang ezyang Have Cabal build a Setup executable and test with it.
I recently discovered that build-tools can be used to specify
dependencies on internal executables (#220).  This means that
we can make a nice improvement to the package-tests test suite:
instead of manually building a Setup.hs script ourselves,
we can add one to the Cabal file for Cabal, and have the
test suite have a build-tools dependency on it.

Perhaps the only objection to this is that the Cabal *library*
now has an executable cabal-setup.  Maybe with some buildable
shenanigans we can make it so that we never build this executable
unless the user asks for it, or the test suite is enabled.
Alternately, cabal-setup is a generally handy executable to have
around.

In fact, we can use this idea to completely kill the LBI hack.
But I'm going to leave that for another patchset, assuming we
like this one.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>
18035b0
@ezyang ezyang added a commit to ezyang/cabal that referenced this issue Mar 31, 2016
@ezyang ezyang Have Cabal build a Setup executable and test with it.
I recently discovered that build-tools can be used to specify
dependencies on internal executables (#220).  This means that
we can make a nice improvement to the package-tests test suite:
instead of manually building a Setup.hs script ourselves,
we can add one to the Cabal file for Cabal, and have the
test suite have a build-tools dependency on it.

Perhaps the only objection to this is that the Cabal *library*
now has an executable cabal-setup.  Maybe with some buildable
shenanigans we can make it so that we never build this executable
unless the user asks for it, or the test suite is enabled.
Alternately, cabal-setup is a generally handy executable to have
around.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>
3523af5
@ezyang ezyang added the ezyang label Mar 31, 2016
@ezyang ezyang added a commit to ezyang/cabal that referenced this issue Mar 31, 2016
@ezyang ezyang Have Cabal build a Setup executable and test with it.
I recently discovered that build-tools can be used to specify
dependencies on internal executables (#220).  This means that
we can make a nice improvement to the package-tests test suite:
instead of manually building a Setup.hs script ourselves,
we can add one to the Cabal file for Cabal, and have the
test suite have a build-tools dependency on it.

Perhaps the only objection to this is that the Cabal *library*
now has an executable cabal-setup.  Maybe with some buildable
shenanigans we can make it so that we never build this executable
unless the user asks for it, or the test suite is enabled.
Alternately, cabal-setup is a generally handy executable to have
around.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>
5db1dc0
@ezyang ezyang added a commit to ezyang/cabal that referenced this issue Mar 31, 2016
@ezyang ezyang Have Cabal build a Setup executable and test with it.
I recently discovered that build-tools can be used to specify
dependencies on internal executables (#220).  This means that
we can make a nice improvement to the package-tests test suite:
instead of manually building a Setup.hs script ourselves,
we can add one to the Cabal file for Cabal, and have the
test suite have a build-tools dependency on it.

Perhaps the only objection to this is that the Cabal *library*
now has an executable cabal-setup.  Maybe with some buildable
shenanigans we can make it so that we never build this executable
unless the user asks for it, or the test suite is enabled.
Alternately, cabal-setup is a generally handy executable to have
around.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>
2603ac6
@ezyang ezyang added a commit to ezyang/cabal that referenced this issue Apr 1, 2016
@ezyang ezyang Have Cabal build a Setup executable and test with it.
I recently discovered that build-tools can be used to specify
dependencies on internal executables (#220).  This means that
we can make a nice improvement to the package-tests test suite:
instead of manually building a Setup.hs script ourselves,
we can add one to the Cabal file for Cabal, and have the
test suite have a build-tools dependency on it.

Perhaps the only objection to this is that the Cabal *library*
now has an executable cabal-setup.  Maybe with some buildable
shenanigans we can make it so that we never build this executable
unless the user asks for it, or the test suite is enabled.
Alternately, cabal-setup is a generally handy executable to have
around.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>
df57fb0
@khumba khumba added a commit to khumba/qtah that referenced this issue Jul 5, 2016
@khumba khumba Improve the dependencies on qtah-generator.
This changes the dependencies on qtah-generator from build-tools to
build-depends, to work with version dependencies and also to ensure
that the generator gets installed, since build-tools doesn't trigger
installation currently (haskell/cabal#220).
1ed421e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment