Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new-build failure with custom-setup when setup-depends don't contain Cabal #4288

Closed
phadej opened this issue Feb 1, 2017 · 11 comments

Comments

Projects
None yet
5 participants
@phadej
Copy link
Collaborator

commented Feb 1, 2017

custom-issue.cabal:

name:                custom-issue
version:             0.1.0.0
license:             BSD3
license-file:        LICENSE
author:              Oleg Grenrus
maintainer:          oleg.grenrus@iki.fi
build-type:          Custom
extra-source-files:  ChangeLog.md
cabal-version:       >=1.10

custom-setup
  setup-depends:
    base, cabal-doctest

library
  exposed-modules:     CustomIssue
  build-depends:       base >=4.9 && <4.10
  default-language:    Haskell2010

Setup.hs:

import Distribution.Extra.Doctest
main = defaultMainWithDoctests "doctests"

Steps to reproduce

cabal new-build

Output

In order, the following will be built (use -v for more details):
 - custom-issue-0.1.0.0 (lib:custom-issue) (first run)
Building custom-issue-0.1.0.0...
Preprocessing library custom-issue-0.1.0.0...
[1 of 1] Compiling CustomIssue      ( CustomIssue.hs, /home/ogre/mess/custom-issue/dist-newstyle/build/x86_64-linux/ghc-8.0.1/custom-issue-0.1.0.0/build/CustomIssue.o )
Creating package registration file:
/home/ogre/mess/custom-issue/dist-newstyle/tmp/package-registration--16283/pkgConf
cabal: Failed to build custom-issue-0.1.0.0. The failure occurred during the
final install step. The exception was:
user error ('/opt/ghc/8.0.1/bin/ghc-pkg' exited with an error:
custom-issue-0.1.0.0: Warning: Unrecognized field indefinite on line 9
custom-issue-0.1.0.0: Warning: haddock-interfaces:
/home/ogre/mess/custom-issue/dist-newstyle/build/x86_64-linux/ghc-8.0.1/custom-issue-0.1.0.0/doc/html/custom-issue/custom-issue.haddock
doesn't exist or isn't a file
custom-issue-0.1.0.0: Warning: haddock-html:
/home/ogre/mess/custom-issue/dist-newstyle/build/x86_64-linux/ghc-8.0.1/custom-issue-0.1.0.0/doc/html/custom-issue
doesn't exist or isn't a directory
custom-issue-0.1.0.0: installed package info from too old version of Cabal
(key field does not match id field)
)

Workaround

  • Add Cabal to setup-depends

Original issue

Reproducible with ghc-8.0.1 and ghc-8.0.2

distributive-0.5.2-e10f1d2339ae8fc2ff95360c5e91f0ace4f1e27add106f7939389ad5ec9ca0aa.
The failure occurred during the final install step. The exception was:
user error ('/opt/ghc/8.0.1/bin/ghc-pkg' exited with an error:
distributive-0.5.2: Warning: Unrecognized field indefinite on line 16
distributive-0.5.2: Warning: haddock-interfaces:
/home/ogre/.cabal/store/ghc-8.0.1/distributive-0.5.2-e10f1d2339ae8fc2ff95360c5e91f0ace4f1e27add106f7939389ad5ec9ca0aa/share/doc/html/distributive.haddock
doesn't exist or isn't a file
distributive-0.5.2: Warning: haddock-html:
/home/ogre/.cabal/store/ghc-8.0.1/distributive-0.5.2-e10f1d2339ae8fc2ff95360c5e91f0ace4f1e27add106f7939389ad5ec9ca0aa/share/doc/html
doesn't exist or isn't a directory
distributive-0.5.2: installed package info from too old version of Cabal (key
field does not match id field)

Steps to reproduce

cabal get comonod-5
cd comonad-5
cabal new-build --disable-tests -j1

/note/ -j1 isn't necessary

Additional info

part of new-build log building the distributive

% /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup/setup register --verbose=2 --builddir=/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2 --inplace --gen-pkg-config=/home/ogre/mess/distributive-0.5.2/dist-newstyle/tmp/package-registration--12792/pkgConf
/opt/ghc/8.0.1/bin/ghc --abi-hash -fbuilding-cabal-package -O -outputdir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -odir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -hidir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -stubdir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -i -i/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -isrc -i/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build/autogen -I/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build/autogen -I/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -optP-include -optP/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build/autogen/cabal_macros.h -this-unit-id distributive-0.5.2-3Qb1gotTB50JIJFSZJyPVw -hide-all-packages -no-user-package-db -package-db /home/ogre/.cabal/store/ghc-8.0.1/package.db -package-db /home/ogre/mess/distributive-0.5.2/dist-newstyle/packagedb/ghc-8.0.1 -package-id base-4.9.0.0 -package-id base-orphans-0.5.4-221596738ac95aba5b36b0b0362492907162ea1af94cf8f335da4c27a8bef6e9 -package-id tagged-0.8.5-6a76d83406e0466a505c9569c11ade18b2bb910a6a09df4ba0d355bf7922badf -package-id transformers-0.5.2.0 -package-id transformers-compat-0.5.1.4-a6f22a40e821e7b9a582fac258c4122bb958675f217a786f78176ce395f17184 -XHaskell98 Data.Distributive Data.Distributive.Generic -Wall
Creating package registration file:
/home/ogre/mess/distributive-0.5.2/dist-newstyle/tmp/package-registration--12792/pkgConf
/home/ogre/mess/distributive-0.5.2/dist-newstyle/tmp/package-registration--12792/:
openBinaryTempFileWithDefaultPermissions: does not exist (No such file or
directory)

That directory doesn't seem to exist.

@phadej

This comment has been minimized.

Copy link
Collaborator Author

commented Feb 1, 2017

does any one remember was there a bug in 1.24?

The GHC-7.10.3 seems successfully register stuff:

/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-7.10.3/distributive-0.5.2/setup/setup
register --verbose=2
--builddir=/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-7.10.3/distributive-0.5.2
--inplace

But otoh, distributive travis did pass yesterday fine, EDIT: because we didn't use new-build there!

@phadej

This comment has been minimized.

Copy link
Collaborator Author

commented Feb 1, 2017

/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup
/opt/ghc/8.0.1/bin/ghc --make -fbuilding-cabal-package -odir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup -hidir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup -i -i/home/ogre/mess/distributive-0.5.2/. -optP-include -optP/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup/setup_macros.h -hide-all-packages -no-user-package-db -package-db /home/ogre/.cabal/store/ghc-8.0.1/package.db -package-db /home/ogre/mess/distributive-0.5.2/dist-newstyle/packagedb/ghc-8.0.1 -package-id base-4.9.0.0 -package-id cabal-doctest-1-6a5408abc48f41d4246a205b4e0e096398ba6620b428170fc35098b9b2458faa /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup/setup.hs -o /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup/setup -threaded
/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup/setup
register --verbose=2
--builddir=/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2
--inplace
--gen-pkg-config=/home/ogre/mess/distributive-0.5.2/dist-newstyle/tmp/package-registration--28640/pkgConf
/opt/ghc/8.0.1/bin/ghc --abi-hash -fbuilding-cabal-package -O -outputdir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -odir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -hidir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -stubdir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -i -i/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -isrc -i/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build/autogen -I/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build/autogen -I/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -optP-include -optP/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build/autogen/cabal_macros.h -this-unit-id distributive-0.5.2-3Qb1gotTB50JIJFSZJyPVw -hide-all-packages -no-user-package-db -package-db /home/ogre/.cabal/store/ghc-8.0.1/package.db -package-db /home/ogre/mess/distributive-0.5.2/dist-newstyle/packagedb/ghc-8.0.1 -package-id base-4.9.0.0 -package-id base-orphans-0.5.4-221596738ac95aba5b36b0b0362492907162ea1af94cf8f335da4c27a8bef6e9 -package-id tagged-0.8.5-6a76d83406e0466a505c9569c11ade18b2bb910a6a09df4ba0d355bf7922badf -package-id transformers-0.5.2.0 -package-id transformers-compat-0.5.1.4-a6f22a40e821e7b9a582fac258c4122bb958675f217a786f78176ce395f17184 -XHaskell98 Data.Distributive Data.Distributive.Generic -Wall
Creating package registration file:
/home/ogre/mess/distributive-0.5.2/dist-newstyle/tmp/package-registration--28640/pkgConf
"register"
InstalledPackageInfo {sourcePackageId = PackageIdentifier {pkgName = PackageName "distributive", pkgVersion = mkVersion [0,5,2]}, installedUnitId = UnitId "distributive-0.5.2-3Qb1gotTB50JIJFSZJyPVw", installedComponentId_ = ComponentId "", instantiatedWith = [], compatPackageKey = "distributive-0.5.2-3Qb1gotTB50JIJFSZJyPVw", license = BSD3, copyright = "Copyright (C) 2011-2016 Edward A. Kmett", maintainer = "Edward A. Kmett <ekmett@gmail.com>", author = "Edward A. Kmett", stability = "provisional", homepage = "http://github.com/ekmett/distributive/", pkgUrl = "", synopsis = "Distributive functors -- Dual to Traversable", description = "Distributive functors -- Dual to Traversable", category = "Data Structures", abiHash = AbiHash "103f62b0c9b3d483d1d59f3559a62b22", indefinite = False, exposed = True, exposedModules = [ExposedModule {exposedName = ModuleName ["Data","Distributive"], exposedReexport = Nothing},ExposedModule {exposedName = ModuleName ["Data","Distributive","Generic"], exposedReexport = Nothing}], hiddenModules = [], trusted = False, importDirs = ["/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build"], libraryDirs = ["/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build"], libraryDynDirs = [], dataDir = "/home/ogre/mess/distributive-0.5.2", hsLibraries = ["HSdistributive-0.5.2-3Qb1gotTB50JIJFSZJyPVw"], extraLibraries = [], extraGHCiLibraries = [], includeDirs = [], includes = [], depends = [UnitId "base-4.9.0.0",UnitId "base-orphans-0.5.4-221596738ac95aba5b36b0b0362492907162ea1af94cf8f335da4c27a8bef6e9",UnitId "tagged-0.8.5-6a76d83406e0466a505c9569c11ade18b2bb910a6a09df4ba0d355bf7922badf",UnitId "transformers-0.5.2.0",UnitId "transformers-compat-0.5.1.4-a6f22a40e821e7b9a582fac258c4122bb958675f217a786f78176ce395f17184"], abiDepends = [], ccOptions = [], ldOptions = [], frameworkDirs = [], frameworks = [], haddockInterfaces = ["/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/doc/html/distributive/distributive.haddock"], haddockHTMLs = ["/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/doc/html/distributive"], pkgRoot = Nothing}
/opt/ghc/8.0.1/bin/ghc-pkg update - --global --no-user-package-db '--package-db=/home/ogre/.cabal/store/ghc-8.0.1/package.db' '--package-db=/home/ogre/mess/distributive-0.5.2/dist-newstyle/packagedb/ghc-8.0.1'
cabal: Failed to build distributive-0.5.2-inplace. The failure occurred during
the final install step. The exception was:
user error ('/opt/ghc/8.0.1/bin/ghc-pkg' exited with an error:
distributive-0.5.2: Warning: Unrecognized field indefinite on line 16
distributive-0.5.2: Warning: haddock-interfaces:
/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/doc/html/distributive/distributive.haddock
doesn't exist or isn't a file
distributive-0.5.2: Warning: haddock-html:
/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/doc/html/distributive
doesn't exist or isn't a directory
distributive-0.5.2: installed package info from too old version of Cabal (key
field does not match id field)

I see there is comment in ProjectBuilding:

                -- We register ourselves rather than via Setup.hs. We need to
                -- grab and modify the InstalledPackageInfo. We decide what
                -- the installed package id is, not the build system.

but why there is both .../setup register and ghc-pkg update ?

ping @dcoutts

EDIT this is Cabal with

diff --git a/cabal-install/Distribution/Client/ProjectBuilding.hs b/cabal-install/Distribution/Client/ProjectBuilding.hs
index 8733dbe..2aa1f69 100644
--- a/cabal-install/Distribution/Client/ProjectBuilding.hs
+++ b/cabal-install/Distribution/Client/ProjectBuilding.hs
@@ -1086,6 +1086,8 @@ buildInplaceUnpackedPackage verbosity
           mipkg <- if elabRequiresRegistration pkg
             then do
                 ipkg0 <- generateInstalledPackageInfo
+                print "register"
+                print ipkg0

not sure where from the first register is even called.

@phadej

This comment has been minimized.

Copy link
Collaborator Author

commented Feb 1, 2017

hmm..

Interestingly, if i cabal get distributive-0.5.2 and add Cabal, filepath, directory to setup-depends, problem seems go away. But those packages aren't directly used in Setup.lhs

EDIT verifying currently with travis build.

@23Skidoo

This comment has been minimized.

Copy link
Member

commented Feb 1, 2017

Just bumped into this as well.

@23Skidoo

This comment has been minimized.

Copy link
Member

commented Feb 1, 2017

I wonder why the setup script compiled without Cabal in build-depends of the custom-setup section, since it seems to be importing stuff from Distribution.*.

@23Skidoo 23Skidoo added the type: bug label Feb 1, 2017

@23Skidoo 23Skidoo added this to the 2.2 milestone Feb 1, 2017

@phadej phadej changed the title new-build failure new-build failure with custom-setup when setup-depends don't contain Cabal Feb 1, 2017

@phadej

This comment has been minimized.

Copy link
Collaborator Author

commented Feb 1, 2017

@23Skidoo edited the original issue with minimal example. if cabal-install knows how to handle setup-depends, Setup.hs doesn't need Cabal, if you check the CPP magic there properly.

@23Skidoo

This comment has been minimized.

Copy link
Member

commented Feb 1, 2017

@phadej Thanks, that clears it.

@ezyang

This comment has been minimized.

Copy link
Contributor

commented Feb 2, 2017

At a guess, this is probably because cabal-install is applying an upper bound to the choice of Cabal library, even when an older version of Cabal wouldn't work with a new GHC? Though I'm a bit confused because 1.24 should be a permitted version in any case.

@mgsloan

This comment has been minimized.

Copy link
Collaborator

commented Nov 13, 2017

Recently encountered something similar in stack - commercialhaskell/stack#3560 (comment) . Should there always be an implicit dependency on Cabal for Setup.hs?

I don't think there should be. What if there is a modified version of Cabal that has a different name? Now the implicit dep on Cabal could cause module ambiguity.

@grayjay

This comment has been minimized.

Copy link
Collaborator

commented Nov 19, 2017

@mgsloan I wasn't able to reproduce this issue, but I don't think cabal-install should add an implicit dependency on Cabal in this case. I think there are only two places where it adds implicit dependencies:

  1. mkDefaultSetupDeps :: UnresolvedSourcePackage -> Maybe [Dependency]
    This is a workaround for #3199. It only applies to old build in certain cases where the .cabal file contains buildable: False.
  2. new-build adds a set of default setup dependencies to packages with build-type: Custom but no custom-setup section. I don't think new-build is ever supposed to modify an existing custom-setup.
@grayjay

This comment has been minimized.

Copy link
Collaborator

commented Nov 19, 2017

I made a mistake before, and I can actually reproduce this bug with custom-issue.cabal. I used cabal from master and GHC 8.0.1, and I compared the result of new-build with and without the explicit dependency on Cabal. I think that the explicit Cabal dependency affected the choice of cabal spec version, not the version of the Cabal dependency. The log showed that cabal chose the same instance of Cabal in both cases, the installed Cabal-1.24.0.0. I also tried printing the spec version in D.C.ProjectPlanning.packageSetupScriptSpecVersion:

packageSetupScriptSpecVersion _ pkg deps =
case find ((cabalPkgname ==) . packageName) (CD.setupDeps deps) of
Just dep -> packageVersion dep
Nothing -> PD.specVersion pkg

With an explicit Cabal dependency, cabal chose that dependency's version (1.24.0.0) at line 2896. Without the direct dependency, cabal chose the package's cabal spec version at line 2897, which was 1.10. I think that packageSetupScriptSpecVersion needs to also check for indirect setup dependencies on Cabal.

grayjay added a commit to grayjay/cabal that referenced this issue Feb 26, 2018

Look for transitive setup dependency on Cabal when choosing Cabal spe…
…c version.

Fixes haskell#4288.

Previously, new-build only looked for a direct dependency on Cabal when choosing
the spec version to use for running the setup script.  When the setup script
only had a transitive dependency on Cabal, cabal used the package's
cabal-version field, which could easily differ from the actual Cabal
dependency's version.  Then cabal could pass flags to the setup script that
weren't recognized by the Cabal dependency.

This commit considers any setup dependency on Cabal when choosing the Cabal spec
version.

grayjay added a commit to grayjay/cabal that referenced this issue Feb 26, 2018

@grayjay grayjay closed this in #5170 Mar 8, 2018

grayjay added a commit to grayjay/cabal that referenced this issue Mar 8, 2018

Look for transitive setup dependency on Cabal when choosing Cabal spe…
…c version.

Fixes haskell#4288.

Previously, new-build only looked for a direct dependency on Cabal when choosing
the spec version to use for running the setup script.  When the setup script
only had a transitive dependency on Cabal, cabal used the package's
cabal-version field, which could easily differ from the actual Cabal
dependency's version.  Then cabal could pass flags to the setup script that
weren't recognized by the Cabal dependency.

This commit considers any setup dependency on Cabal when choosing the Cabal spec
version.

(cherry picked from commit ea01b9d)

grayjay added a commit to grayjay/cabal that referenced this issue Mar 8, 2018

Add a regression test for issue haskell#4288.
(cherry picked from commit f4f382a)

Sonna added a commit to Sonna/toy-robot-haskell that referenced this issue Mar 29, 2018

Fix expose of `Haq` module within haskell spec
Fix the setup of this Haskell project within CircleCI and when setting
up on a new machine or within Docker. This required that `Haq` not be
considered a Library (without an executable), but instead an exposed
module that was built and included within both the base project, as well
as its tests.

This should resolve problems that where occurring when running
`cabal install`

== Notes:
- Change the order of `cabal update` with `cabal sandbox init`, since
  update will only run it there is a sandbox available, which is not
  present in the Haskell image container

- Add `--enable-tests` and `--enable-documentation` to cabal install to
  slience other warnings

== References:
- [How to write a Haskell program \- HaskellWiki]
  (https://wiki.haskell.org/How_to_write_a_Haskell_program#Add_some_tests)

- [cabal \- Set up Haskell Project and run tests \- Stack Overflow]
  (https://stackoverflow.com/questions/38928802/set-up-haskell-project-and-run-tests)

- [Cabal\-Install \- HaskellWiki]
  (https://wiki.haskell.org/Cabal-Install)

- [Invoking Haddock — Haddock 1\.0 documentation]
  (http://haskell-haddock.readthedocs.io/en/latest/invoking.html)

- [new\-build failure with custom\-setup when setup\-depends don't
  contain Cabal · Issue \#4288 · haskell/cabal]
  (haskell/cabal#4288)

- [2\.1\. Configuration — Cabal <release> User's Guide]
  (https://www.haskell.org/cabal/users-guide/installing-packages.html)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.