Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to configure 'entropy' when depending on newer 'unix' #1110

Closed
hesselink opened this issue Oct 6, 2015 · 25 comments
Closed

Unable to configure 'entropy' when depending on newer 'unix' #1110

hesselink opened this issue Oct 6, 2015 · 25 comments
Assignees
Milestone

Comments

@hesselink
Copy link
Contributor

If I (in a custom snapshot) depend on entropy, and also depend on a newer version of unix than comes with ghc 7.8, the configure fails:

$ stack build
entropy-0.3.7: configure
Progress: 1/2
--  While building package entropy-0.3.7 using:
      /usr/bin/runhaskell -clear-package-db -global-package-db -package-db=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/pkgdb/ -package-db=/Users/erik/temp/test/.stack-work/install/x86_64-osx/custom-custom-test/7.8.3/pkgdb/ -hide-all-packages -package=Cabal-1.18.1.3 -package-id=array-0.5.0.0-98aa445e59f3eb0c886795ff07406d84 -package-id=base-4.7.0.1-c64d224738ec7af4085e89ca9c12c37b -package-id=binary-0.7.1.0-06fe2cfe2c00e6f7d43cdb53345c4945 -package-id=bytestring-0.10.4.0-18fe2f3ce284617c82da1702e16772cf -package-id=containers-0.5.5.1-23e2a2b94d6e452c773209f31d8672c5 -package-id=deepseq-1.3.0.2-733fe43e121f761739636bd6670bea77 -package-id=filepath-1.3.0.2-1580a61d3226e4be45fe2130dc2881e3 -package-id=ghc-prim-0.3.1.0-954cb57749cf319beafdc89b3415422c -package-id=hoopl-3.10.0.1-78e45fe2aae96846315ab833b31e7409 -package-id=integer-gmp-0.5.1.0-d42e6a7874a019e6a0d1c7305ebc83c4 -package-id=mtl-2.1.3.1-8bcc0591131896cfc8761a93703d4c61 -package-id=old-locale-1.0.0.6-09baf1dbc5be8338e5eba7c5bb515505 -package-id=old-time-1.1.0.2-eab97c1e6d2cda84575b30ce4bfcfaf2 -package-id=parsec-3.1.9-26e7a52d0873abf27275d79645acd443 -package-id=pretty-1.1.1.1-055fb3b6b88c48ad1bf91e5cf35828d1 -package-id=template-haskell-2.9.0.0-310613e57f38a482326cc4ba2989009e -package-id=terminfo-0.4.0.0-b02cf5922e12c1a9557df1270cdf18b7 -package-id=text-1.2.0.6-e9cc2f9cbc7fea487488c8ed44ced4c0 -package-id=time-1.4.2-bf925e935c287d0b75398fe297453c28 -package-id=transformers-0.3.0.0-16a97696ae672940f1523b262e566ba5 -package-id=unix-2.7.1.0-184e782f6f9d2dc43807de1f0533a4d1 -package-id=xhtml-3000.2.1-2da8c2cd0581a1aa5b8f1c06f5c3b943 -package-id=zlib-0.5.4.2-4ebdf7bad9192e1149e8553ecd1c3649 /var/folders/qg/4lbbbp0d1hbg7f6_cr2r8h8h0000gn/T/stack58618/entropy-0.3.7/Setup.hs --builddir=.stack-work/dist/x86_64-osx/Cabal-1.18.1.3/ configure --user --package-db=clear --package-db=global --package-db=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/pkgdb/ --libdir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/lib --bindir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/bin --datadir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/share --libexecdir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/libexec --sysconfdir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/etc --docdir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/doc/entropy-0.3.7 --htmldir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/doc/entropy-0.3.7 --haddockdir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/doc/entropy-0.3.7 --constraint=base==4.7.0.1 --constraint=bytestring==0.10.4.0 --constraint=unix==2.7.1.0
    Process exited with code: ExitFailure 1
    Logs have been written to: /Users/erik/temp/test/.stack-work/logs/entropy-0.3.7.log


    /var/folders/qg/4lbbbp0d1hbg7f6_cr2r8h8h0000gn/T/stack58618/entropy-0.3.7/Setup.hs:10:8:
        Could not find module ‘System.Process’
        It is a member of the hidden package ‘process-1.2.0.0’.
        Use -v to see a list of the files searched for.

    /var/folders/qg/4lbbbp0d1hbg7f6_cr2r8h8h0000gn/T/stack58618/entropy-0.3.7/Setup.hs:11:8:
        Could not find module ‘System.Directory’
        It is a member of the hidden package ‘directory-1.2.1.0’.
        Use -v to see a list of the files searched for.

To reproduce, download this reproduction and issue stack build.

@snoyberg
Copy link
Contributor

snoyberg commented Oct 6, 2015

Would you be able to try upgrading to Cabal 1.22? This looks similar to #1090.

@hesselink
Copy link
Contributor Author

I just tried, but I get the same error:

$ cabal install --global Cabal
Resolving dependencies...
Configuring Cabal-1.22.4.0...
Building Cabal-1.22.4.0...
Installed Cabal-1.22.4.0
$ rm -r .stack-work/ ~/.stack/snapshots/x86_64-osx/custom-custom-test/
$ stack build
unix-2.7.1.0: configure
unix-2.7.1.0: build
unix-2.7.1.0: install
entropy-0.3.7: configure
Progress: 2/3
--  While building package entropy-0.3.7 using:
      /usr/bin/runhaskell -clear-package-db -global-package-db -package-db=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/pkgdb/ -package-db=/Users/erik/temp/test/.stack-work/install/x86_64-osx/custom-custom-test/7.8.3/pkgdb/ -hide-all-packages -package=Cabal-1.22.4.0 -package-id=array-0.5.0.0-98aa445e59f3eb0c886795ff07406d84 -package-id=base-4.7.0.1-c64d224738ec7af4085e89ca9c12c37b -package-id=binary-0.7.1.0-06fe2cfe2c00e6f7d43cdb53345c4945 -package-id=bytestring-0.10.4.0-18fe2f3ce284617c82da1702e16772cf -package-id=containers-0.5.5.1-23e2a2b94d6e452c773209f31d8672c5 -package-id=deepseq-1.3.0.2-733fe43e121f761739636bd6670bea77 -package-id=filepath-1.3.0.2-1580a61d3226e4be45fe2130dc2881e3 -package-id=ghc-prim-0.3.1.0-954cb57749cf319beafdc89b3415422c -package-id=hoopl-3.10.0.1-78e45fe2aae96846315ab833b31e7409 -package-id=integer-gmp-0.5.1.0-d42e6a7874a019e6a0d1c7305ebc83c4 -package-id=mtl-2.1.3.1-8bcc0591131896cfc8761a93703d4c61 -package-id=old-locale-1.0.0.6-09baf1dbc5be8338e5eba7c5bb515505 -package-id=old-time-1.1.0.2-eab97c1e6d2cda84575b30ce4bfcfaf2 -package-id=parsec-3.1.9-26e7a52d0873abf27275d79645acd443 -package-id=pretty-1.1.1.1-055fb3b6b88c48ad1bf91e5cf35828d1 -package-id=template-haskell-2.9.0.0-310613e57f38a482326cc4ba2989009e -package-id=terminfo-0.4.0.0-b02cf5922e12c1a9557df1270cdf18b7 -package-id=text-1.2.0.6-e9cc2f9cbc7fea487488c8ed44ced4c0 -package-id=time-1.4.2-bf925e935c287d0b75398fe297453c28 -package-id=transformers-0.3.0.0-16a97696ae672940f1523b262e566ba5 -package-id=unix-2.7.1.0-184e782f6f9d2dc43807de1f0533a4d1 -package-id=xhtml-3000.2.1-2da8c2cd0581a1aa5b8f1c06f5c3b943 -package-id=zlib-0.5.4.2-4ebdf7bad9192e1149e8553ecd1c3649 /var/folders/qg/4lbbbp0d1hbg7f6_cr2r8h8h0000gn/T/stack76334/entropy-0.3.7/Setup.hs --builddir=.stack-work/dist/x86_64-osx/Cabal-1.22.4.0/ configure --user --package-db=clear --package-db=global --package-db=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/pkgdb/ --libdir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/lib --bindir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/bin --datadir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/share --libexecdir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/libexec --sysconfdir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/etc --docdir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/doc/entropy-0.3.7 --htmldir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/doc/entropy-0.3.7 --haddockdir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/doc/entropy-0.3.7 --dependency=base=base-4.7.0.1-c64d224738ec7af4085e89ca9c12c37b --dependency=bytestring=bytestring-0.10.4.0-18fe2f3ce284617c82da1702e16772cf --dependency=unix=unix-2.7.1.0-184e782f6f9d2dc43807de1f0533a4d1
    Process exited with code: ExitFailure 1
    Logs have been written to: /Users/erik/temp/test/.stack-work/logs/entropy-0.3.7.log


    /var/folders/qg/4lbbbp0d1hbg7f6_cr2r8h8h0000gn/T/stack76334/entropy-0.3.7/Setup.hs:10:8:
        Could not find module ‘System.Process’
        It is a member of the hidden package ‘process-1.2.0.0’.
        Use -v to see a list of the files searched for.

    /var/folders/qg/4lbbbp0d1hbg7f6_cr2r8h8h0000gn/T/stack76334/entropy-0.3.7/Setup.hs:11:8:
        Could not find module ‘System.Directory’
        It is a member of the hidden package ‘directory-1.2.1.0’.
        Use -v to see a list of the files searched for.

@hesselink
Copy link
Contributor Author

Note that if you look at the actual command that is run, it doesn't specify the package ids for process and directory. If you compare with the command when unix isn't specified as a dependency, those package ids appear and the command succeeds.

@snoyberg
Copy link
Contributor

snoyberg commented Oct 6, 2015

Ah, good catch, this is that annoying issue of the packages provided to
Setup.hs itself again.

On Tue, Oct 6, 2015 at 1:05 PM, Erik Hesselink notifications@github.com
wrote:

Note that if you look at the actual command that is run, it doesn't
specify the package ids for process and directory. If you compare with the
command when unix isn't specified as a dependency, those package ids
appear and the command succeeds.


Reply to this email directly or view it on GitHub
#1110 (comment)
.

@hesselink
Copy link
Contributor Author

Yeah, hopefully this will get better once Setup.hs build-depends become more widely used, but for now, I guess I'll just keep filing issues as I run into them :/

@snoyberg
Copy link
Contributor

snoyberg commented Oct 6, 2015

Can you confirm that manually editing the entropy package .cabal file to add process and directory as build-depends fixes the problem?

@hesselink
Copy link
Contributor Author

I don't think that feature is even in a released Cabal version yet. Or do you mean just adding it to the normal library build-depends?

@snoyberg
Copy link
Contributor

snoyberg commented Oct 6, 2015

Yes, that's what I meant.

@hesselink
Copy link
Contributor Author

Hmm, that gives more errors. Here's what I did:

  • Remove entropy-0.3.7 from the snapshot.
  • cabal get entropy-0.3.7.
  • Add the following to packages in stack.yaml:
    • location: entropy-0.3.7
      extra-dep: true
  • Add the following to snapshot.yaml:
    • directory-1.2.1.0
    • process-1.2.0.0
  • Run stack build
  • Get the error below.
  • Remove the snapshot and .stack-work.
  • Try again, get the same error:
$ stack build
unix-2.7.1.0: configure
unix-2.7.1.0: build
unix-2.7.1.0: install
directory-1.2.1.0: configure
directory-1.2.1.0: build
directory-1.2.1.0: install
process-1.2.0.0: configure
Progress: 3/5
--  While building package process-1.2.0.0 using:
      /usr/bin/runhaskell -clear-package-db -global-package-db -package-db=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/pkgdb/ -package-db=/Users/erik/temp/test/.stack-work/install/x86_64-osx/custom-custom-test/7.8.3/pkgdb/ -hide-all-packages -package=Cabal-1.18.1.3 -package-id=array-0.5.0.0-98aa445e59f3eb0c886795ff07406d84 -package-id=base-4.7.0.1-c64d224738ec7af4085e89ca9c12c37b -package-id=binary-0.7.1.0-06fe2cfe2c00e6f7d43cdb53345c4945 -package-id=bytestring-0.10.4.0-18fe2f3ce284617c82da1702e16772cf -package-id=containers-0.5.5.1-23e2a2b94d6e452c773209f31d8672c5 -package-id=deepseq-1.3.0.2-733fe43e121f761739636bd6670bea77 -package-id=directory-1.2.1.0-16828b8d6269b7957cc51e42bc4950f3 -package-id=filepath-1.3.0.2-1580a61d3226e4be45fe2130dc2881e3 -package-id=ghc-prim-0.3.1.0-954cb57749cf319beafdc89b3415422c -package-id=hoopl-3.10.0.1-78e45fe2aae96846315ab833b31e7409 -package-id=integer-gmp-0.5.1.0-d42e6a7874a019e6a0d1c7305ebc83c4 -package-id=mtl-2.1.3.1-8bcc0591131896cfc8761a93703d4c61 -package-id=old-locale-1.0.0.6-09baf1dbc5be8338e5eba7c5bb515505 -package-id=old-time-1.1.0.2-eab97c1e6d2cda84575b30ce4bfcfaf2 -package-id=parsec-3.1.9-26e7a52d0873abf27275d79645acd443 -package-id=pretty-1.1.1.1-055fb3b6b88c48ad1bf91e5cf35828d1 -package-id=template-haskell-2.9.0.0-310613e57f38a482326cc4ba2989009e -package-id=terminfo-0.4.0.0-b02cf5922e12c1a9557df1270cdf18b7 -package-id=text-1.2.0.6-e9cc2f9cbc7fea487488c8ed44ced4c0 -package-id=time-1.4.2-bf925e935c287d0b75398fe297453c28 -package-id=transformers-0.3.0.0-16a97696ae672940f1523b262e566ba5 -package-id=unix-2.7.1.0-184e782f6f9d2dc43807de1f0533a4d1 -package-id=xhtml-3000.2.1-2da8c2cd0581a1aa5b8f1c06f5c3b943 -package-id=zlib-0.5.4.2-4ebdf7bad9192e1149e8553ecd1c3649 /var/folders/qg/4lbbbp0d1hbg7f6_cr2r8h8h0000gn/T/stack25148/process-1.2.0.0/Setup.hs --builddir=.stack-work/dist/x86_64-osx/Cabal-1.18.1.3/ configure --user --package-db=clear --package-db=global --package-db=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/pkgdb/ --libdir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/lib --bindir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/bin --datadir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/share --libexecdir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/libexec --sysconfdir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/etc --docdir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/doc/process-1.2.0.0 --htmldir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/doc/process-1.2.0.0 --haddockdir=/Users/erik/.stack/snapshots/x86_64-osx/custom-custom-test/7.8.3/doc/process-1.2.0.0 --constraint=base==4.7.0.1 --constraint=deepseq==1.3.0.2 --constraint=directory==1.2.1.0 --constraint=filepath==1.3.0.2 --constraint=unix==2.7.1.0
    Process exited with code: ExitFailure 1
    Logs have been written to: /Users/erik/temp/test/.stack-work/logs/process-1.2.0.0.log

    <command line>: cannot satisfy -package Cabal-1.18.1.3:
        Cabal-1.18.1.3-89fcb17b479617112f670ab32d335f43 is unusable due to missing or recursive dependencies:
          directory-1.2.1.0-bfdf4eb032907ecaf13c26632da610b8 process-1.2.0.0-4870cdb503fdbb301e26c7b926e216b2
        (use -v for more information)

Should I file that as a separate bug?

Then I switched to the latest versions of directory and process (directory-1.2.4.0, process-1.3.0.0), but stack build gave the same error. After removing .stack-work and the snapshot again, it finally worked.

@snoyberg
Copy link
Contributor

snoyberg commented Oct 6, 2015

I don't think it's worth opening that as an issue, especially since #863 (which I was planning on starting on today) will have a major impact on it. The question then comes down to this issue with entropy. Do we consider Setup.hs being custom and relying on an unlisted global and overridden package something that Stack needs to deal with, or state that modifying entropy like this is the right solution? I really don't know.

Pinging @mitchellwrosen for input as well.

@snoyberg snoyberg self-assigned this Oct 6, 2015
@snoyberg snoyberg added this to the P1: Must milestone Oct 6, 2015
@hesselink
Copy link
Contributor Author

At the very least it's confusing that upgrading a seemingly unrelated package (unix) suddenly hides these two packages from the Setup.hs of entropy. So whatever you decide, I think the packages should either always be available, or never (but that would probably break all the things).

@snoyberg
Copy link
Contributor

snoyberg commented Oct 6, 2015

I'd also prefer never in an ideal world. But for the "always" case to work, it seems like what we'd need to do is that, for every package with a custom Setup.hs, add an implicit directory, process, etc dependency to ensure that they are available against the selected list of dependencies. In fact, if we make that configurable (custom-implicit-deps: [process, directory]), maybe that's the right solution, at least until explicit Setup.hs dependencies becomes widespread. As ugly as that sounds, what do you think?

@hesselink
Copy link
Contributor Author

Why would you need to make them available? These are the packages that come with ghc, they should always be available. The problem seems to be that if there is one in the global db, and one in the project db, it won't be passed. Is that correct? If so, then perhaps you could pick the set of 'core' packages, find the right version (global, snapshot, project) for each one, and pass those versions. Since there should always be a version of a package, that should work, right?

@snoyberg
Copy link
Contributor

snoyberg commented Oct 6, 2015

No. Consider the case (which actually occurred in Stackage) where:

  • Package X from the global database is replaced
  • Package Y depends on X
  • Setup.hs uses both X (and requires the modified X) and Y

If you use just the global packages, you'll get the unmodified X. If you use the modified X and Y from the global database, you'll get mismatched types. The only option is to recompile Y against the modified X and then use that.

@hesselink
Copy link
Contributor Author

Ah, I see. Yeah, in that case adding these dependencies seems good. I don't see the use case for making it configurable (the set that comes with ghc seems good enough) but it probably doesn't hurt, there's always some corner case out there.

@snoyberg
Copy link
Contributor

snoyberg commented Oct 6, 2015

Actually, I really can't see a way to implement this that doesn't introduce some kind of major regression for someone. At the very least, it can trigger a whole bunch of unnecessary package installs. I'm beginning to think there's no way to handle this case gracefully, and just documenting the workaround of locally checking out the relevant packages and moving on.

@hesselink
Copy link
Contributor Author

Hmm, that doesn't sound like a great workaround. We're running into this now, and are unable to install our snapshot from scratch now it seems. I've seen at least two packages in our dependencies where we're running into this, and there might be more. I'd rather not maintain lots of forks like that. Perhaps there is a different workaround? Something like keeping all (most?) core packages at their original versions?

@snoyberg
Copy link
Contributor

snoyberg commented Oct 6, 2015

How about an option in stack.yaml that says "always use the global package
database in its entirety for Setup.hs, and ignore everything from snapshot
and local databases?" The option could even have a default and a
per-package override.

On Tue, Oct 6, 2015 at 3:01 PM, Erik Hesselink notifications@github.com
wrote:

Hmm, that doesn't sound like a great workaround. We're running into this
now, and are unable to install our snapshot from scratch now it seems. I've
seen at least two packages in our dependencies where we're running into
this, and there might be more. I'd rather not maintain lots of forks like
that. Perhaps there is a different workaround? Something like keeping all
(most?) core packages at their original versions?


Reply to this email directly or view it on GitHub
#1110 (comment)
.

@hesselink
Copy link
Contributor Author

That sounds perfect, but I understood from the above that that wasn't possible. I think that should cover most (all?) dependencies for Setup.hs, since anything else would be very tricky to get to work with cabal as well. What would the default of that option be? And what would the behavior be when it wasn't enabled?

@snoyberg
Copy link
Contributor

snoyberg commented Oct 6, 2015

It will work in many common cases, just not every case. The more unusual
situation here is overriding a global package, not a wonky Setup.hs file,
so I think the default behavior will remain as-is, and with this option
turned on we'll enact the functionality I just described.

On Tue, Oct 6, 2015 at 3:33 PM, Erik Hesselink notifications@github.com
wrote:

That sounds perfect, but I understood from the above that that wasn't
possible. I think that should cover most (all?) dependencies for Setup.hs,
since anything else would be very tricky to get to work with cabal as well.
What would the default of that option be? And what would the behavior be
when it wasn't enabled?


Reply to this email directly or view it on GitHub
#1110 (comment)
.

@snoyberg
Copy link
Contributor

snoyberg commented Oct 6, 2015

I've just pushed a commit to master with the new explicit-setup-deps configuration option. @hesselink can you give it a shot?

@hesselink
Copy link
Contributor Author

Yes, works like a charm! Both setting explicit-setup-deps to { "entropy" : false } or to { "*" : false } does the trick. I think we'll go with the latter for our project, since I don't imagine any publicly released packages will depend on anything other than the global package db in their Setup.hs.

@snoyberg
Copy link
Contributor

snoyberg commented Oct 8, 2015

Awesome! As a heads-up: I ended up changing the default for this from true to false (see #1025), since it seems to be causing quite a bit of trouble elsewhere too. Pinging @mitchellwrosen about that, you may need to add explicit-setup-deps: { "*" : true } to your stack.yaml.

@hesselink
Copy link
Contributor Author

Good to know, that means we won't have to change our configs at all, and things will just magically work :)

@mitchellwrosen
Copy link
Contributor

Just caught up on this thread, thanks @snoyberg. I definitely didn't realize up front what a nightmare allowing Setup.hs to use local packages would become.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants