New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dependency not getting installed in .stack-work with stack 1.3.0 #2849

Closed
wolftune opened this Issue Dec 15, 2016 · 25 comments

Comments

Projects
None yet
4 participants
@wolftune
Contributor

wolftune commented Dec 15, 2016

related to #2845 this is specifically occurring with http://github.com/snowdriftcoop/snowdrift

Tested on multiple systems (all Linux-based)

With Stack 1.2.0, everything is fine. With Stack 1.3.0, the yesod binary is not found. It's not getting put into .stack-work correctly.

If I build successfully with stack 1.2.0, I can keep the stack-work, then build with stack 1.3.0 and still have things work because the binary built before is still present.

This probably relates to snowdrift using an alternate forked version of yesod-bin with a git address. It seems stack 1.3.0 has a bug where it isn't installing binaries in stack-work correctly when the binary comes from an extra dependency…

@mitchellwrosen

This comment has been minimized.

Contributor

mitchellwrosen commented Dec 15, 2016

Hmm... I filed #2845, but I'm not sure it's related (or even a bug!). I was using 1.2.1, as noted in the ticket.

@mitchellwrosen

This comment has been minimized.

Contributor

mitchellwrosen commented Dec 15, 2016

However, I can reproduce this bug using 1.3; here's how:

$ git clone http://github.com/snowdriftcoop/snowdrift
$ cd snowdrift
$ stack build yesod-bin
$ stack exec -- which yesod

Expected output: yesod installed somewhere in .stack-work/
Actual output: nothing

(I also tried stack build yesod-bin:exe:yesod - no dice)

@mitchellwrosen

This comment has been minimized.

Contributor

mitchellwrosen commented Dec 15, 2016

It also doesn't have to do with the forked yesod-bin package, as pointing at the actual repo at the proper tag (as below) exhibits the same behavior.

- location:
    git: https://github.com/yesodweb/yesod
    commit: d1495bad85565f75ce0238545dbec1d4257aba24
  subdirs:
    - yesod-bin
  extra-dep: true
@mitchellwrosen

This comment has been minimized.

Contributor

mitchellwrosen commented Dec 15, 2016

Here's a log of stack build -v yesod-bin:exe:yesod --no-time-in-log:

Version 1.3.0 x86_64 hpack-0.15.0
[debug] Checking for project config at: /home/mitchell/snowdrift/stack.yaml
@(Stack/Config.hs:863:9)
[debug] Loading project config file stack.yaml
@(Stack/Config.hs:881:13)
[debug] Trying to decode /home/mitchell/.stack/build-plan-cache/x86_64-linux/lts-6.17.cache
@(Data/Store/VersionTagged.hs:68:5)
[debug] Success decoding /home/mitchell/.stack/build-plan-cache/x86_64-linux/lts-6.17.cache
@(Data/Store/VersionTagged.hs:72:13)
[debug] Run process: /sbin/ldconfig -p
@(System/Process/Read.hs:306:3)
[debug] Process finished in 0ms: /sbin/ldconfig -p
@(System/Process/Read.hs:306:3)
[debug] Run process: /usr/bin/gcc -v
@(System/Process/Read.hs:306:3)
[debug] Process finished in 0ms: /usr/bin/gcc -v
@(System/Process/Read.hs:306:3)
[debug] PIE enabled
@(Stack/Setup.hs:574:17)
[debug] Found shared library libtinfo.so.5 in /usr/lib
@(Stack/Setup.hs:563:38)
[debug] Found shared library libtinfo.so.6 in /usr/lib
@(Stack/Setup.hs:563:38)
[debug] Found shared library libncursesw.so.6 in 'ldconfig -p' output
@(Stack/Setup.hs:550:29)
[debug] Found shared library libgmp.so.10 in 'ldconfig -p' output
@(Stack/Setup.hs:550:29)
[debug] Did not find shared library libgmp.so.3
@(Stack/Setup.hs:564:38)
[debug] Using standard GHC build
@(Stack/Setup.hs:597:9)
[debug] Getting global package database location
@(Stack/GhcPkg.hs:55:5)
[debug] Run process: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:306:3)
[debug] Asking GHC for its version
@(Stack/Setup/Installed.hs:103:13)
[debug] Getting Cabal package version
@(Stack/GhcPkg.hs:170:5)
[debug] Run process: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc --numeric-version
@(System/Process/Read.hs:306:3)
[debug] Run process: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Read.hs:306:3)
[debug] Process finished in 13ms: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:306:3)
[debug] Process finished in 12ms: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Read.hs:306:3)
[debug] Process finished in 23ms: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc --numeric-version
@(System/Process/Read.hs:306:3)
[debug] Resolving package entries
@(Stack/Setup.hs:252:5)
[debug] Starting to execute command inside EnvConfig
@(Stack/Runners.hs:163:18)
[debug] Parsing the cabal files of the local packages
@(Stack/Build/Source.hs:298:5)
[debug] Parsing the targets
@(Stack/Build/Source.hs:235:5)
[debug] Exception ignored when attempting to load /home/mitchell/snowdrift/website/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: /home/mitchell/snowdrift/website/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: openBinaryFile: does not exist (No such file or directory)
@(Data/Store/VersionTagged.hs:86:9)
[debug] Start: getPackageFiles /home/mitchell/snowdrift/website/Snowdrift.cabal
@(Stack/Package.hs:250:21)
[debug] Finished in 6ms: getPackageFiles /home/mitchell/snowdrift/website/Snowdrift.cabal
@(Stack/Package.hs:250:21)
[debug] Exception ignored when attempting to load /home/mitchell/snowdrift/crowdmatch/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: /home/mitchell/snowdrift/crowdmatch/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: openBinaryFile: does not exist (No such file or directory)
@(Data/Store/VersionTagged.hs:86:9)
[debug] Start: getPackageFiles /home/mitchell/snowdrift/crowdmatch/crowdmatch.cabal
@(Stack/Package.hs:250:21)
[debug] Finished in 1ms: getPackageFiles /home/mitchell/snowdrift/crowdmatch/crowdmatch.cabal
@(Stack/Package.hs:250:21)
[debug] Exception ignored when attempting to load /home/mitchell/snowdrift/.stack-work/downloaded/6nk1N5bfncwY/persistent/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: /home/mitchell/snowdrift/.stack-work/downloaded/6nk1N5bfncwY/persistent/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: openBinaryFile: does not exist (No such file or directory)
@(Data/Store/VersionTagged.hs:86:9)
[debug] Start: getPackageFiles /home/mitchell/snowdrift/.stack-work/downloaded/6nk1N5bfncwY/persistent/persistent.cabal
@(Stack/Package.hs:250:21)
[debug] Finished in 4ms: getPackageFiles /home/mitchell/snowdrift/.stack-work/downloaded/6nk1N5bfncwY/persistent/persistent.cabal
@(Stack/Package.hs:250:21)
[debug] Exception ignored when attempting to load /home/mitchell/snowdrift/.stack-work/downloaded/Nl5BUGxz7nps/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: /home/mitchell/snowdrift/.stack-work/downloaded/Nl5BUGxz7nps/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: openBinaryFile: does not exist (No such file or directory)
@(Data/Store/VersionTagged.hs:86:9)
[debug] Start: getPackageFiles /home/mitchell/snowdrift/.stack-work/downloaded/Nl5BUGxz7nps/postgresql-simple-migration.cabal
@(Stack/Package.hs:250:21)
[debug] Finished in 4ms: getPackageFiles /home/mitchell/snowdrift/.stack-work/downloaded/Nl5BUGxz7nps/postgresql-simple-migration.cabal
@(Stack/Package.hs:250:21)
[debug] Exception ignored when attempting to load /home/mitchell/snowdrift/run-persist/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: /home/mitchell/snowdrift/run-persist/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: openBinaryFile: does not exist (No such file or directory)
@(Data/Store/VersionTagged.hs:86:9)
[debug] Start: getPackageFiles /home/mitchell/snowdrift/run-persist/run-persist.cabal
@(Stack/Package.hs:250:21)
[debug] Finished in 0ms: getPackageFiles /home/mitchell/snowdrift/run-persist/run-persist.cabal
@(Stack/Package.hs:250:21)
[debug] Exception ignored when attempting to load /home/mitchell/snowdrift/.stack-work/downloaded/bnIHMPumBlP2/yesod-bin/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: /home/mitchell/snowdrift/.stack-work/downloaded/bnIHMPumBlP2/yesod-bin/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: openBinaryFile: does not exist (No such file or directory)
@(Data/Store/VersionTagged.hs:86:9)
[debug] Start: getPackageFiles /home/mitchell/snowdrift/.stack-work/downloaded/bnIHMPumBlP2/yesod-bin/yesod-bin.cabal
@(Stack/Package.hs:250:21)
[debug] Finished in 2ms: getPackageFiles /home/mitchell/snowdrift/.stack-work/downloaded/bnIHMPumBlP2/yesod-bin/yesod-bin.cabal
@(Stack/Package.hs:250:21)
[debug] Finding out which packages are already installed
@(Stack/Build/Installed.hs:68:5)
[debug] Run process: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --global --no-user-package-db dump --expand-pkgroot
@(System/Process/Read.hs:306:3)
[debug] Process finished in 15ms: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --global --no-user-package-db dump --expand-pkgroot
@(System/Process/Read.hs:306:3)
[debug] Ignoring package haskeline due to wanting version 0.7.2.3 instead of 0.7.2.1
@(Stack/Build/Installed.hs:191:5)
[debug] Ignoring package terminfo due to wanting version 0.4.0.2 instead of 0.4.0.1
@(Stack/Build/Installed.hs:191:5)
[debug] Ignoring package Cabal due to wanting version 1.22.8.0 instead of 1.22.5.0
@(Stack/Build/Installed.hs:191:5)
[debug] Run process: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /home/mitchell/.stack/snapshots/x86_64-linux/lts-6.17/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Read.hs:306:3)
[debug] Process finished in 46ms: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /home/mitchell/.stack/snapshots/x86_64-linux/lts-6.17/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Read.hs:306:3)
[debug] Run process: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /home/mitchell/snowdrift/.stack-work/install/x86_64-linux/lts-6.17/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Read.hs:306:3)
[debug] Process finished in 11ms: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /home/mitchell/snowdrift/.stack-work/install/x86_64-linux/lts-6.17/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Read.hs:306:3)
[debug] Trying to decode /home/mitchell/.stack/indices/Hackage/00-index.cache
@(Data/Store/VersionTagged.hs:68:5)
[debug] Success decoding /home/mitchell/.stack/indices/Hackage/00-index.cache
@(Data/Store/VersionTagged.hs:72:13)
[debug] Constructing the build plan
@(Stack/Build/ConstructPlan.hs:159:5)
[debug] Checking if we are going to build multiple executables with the same name
@(Stack/Build.hs:196:5)
[debug] Executing the build plan
@(Stack/Build/Execute.hs:454:5)
[debug] Getting global package database location
@(Stack/GhcPkg.hs:55:5)
[debug] Run process: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:306:3)
[debug] Process finished in 11ms: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:306:3)
@mitchellwrosen

This comment has been minimized.

Contributor

mitchellwrosen commented Dec 16, 2016

I think this behavior changed in 58d4f42, and it was intentional.

The solution for the snowdrift guys (cc @wolftune) is to simply set extra-dep: false for the yesod-bin local package.

@mgsloan

This comment has been minimized.

Collaborator

mgsloan commented Dec 17, 2016

Alternatively, require that stack build yesod-bin is run. Sounds like this has been resolved, thanks for investigating @mitchellwrosen !

@mgsloan mgsloan closed this Dec 17, 2016

@mitchellwrosen

This comment has been minimized.

Contributor

mitchellwrosen commented Dec 17, 2016

@mgsloan No problem, but can you clarify what you mean by that last comment?

@wolftune

This comment has been minimized.

Contributor

wolftune commented Dec 17, 2016

@mgsloan the source of the issue has been identified, but I'm not sure that we know that the way the behavior happens is indeed the desired result. At any rate, your understanding is not correct because stack build yesod-bin does not succeed in our case in getting past this issue!

I'm testing right now to see if workaround from @mitchellwrosen succeeds here. Again, not sure that it's the correct answer or a workaround for something that isn't right in stack.

@mitchellwrosen

This comment has been minimized.

Contributor

mitchellwrosen commented Dec 17, 2016

@wolftune It's not a workaround; as I said the change to not treat packages with extra-dep: true as targets was intentional =)

@wolftune

This comment has been minimized.

Contributor

wolftune commented Dec 17, 2016

Confirmed resolution from @mitchellwrosen — but I don't myself understand the meaning here of extra-dep: false versus extra-dep: true and why it would (or wouldn't?) make sense to include a location in stack.yaml that isn't an extra-dep. Clearly, I did not follow the recent changes to stack and the .yaml syntax.

I agree this is thus solved, as far as I can tell. Perhaps the erroneous suggestion from @mgsloan is just a red herring.

@mitchellwrosen

This comment has been minimized.

Contributor

mitchellwrosen commented Dec 17, 2016

Well, looking over the commit message I don't actually know for sure if @snoyberg realized he was changing the behavior of stack build <local dep with executable component>.

@wolftune as far as I know, you have a forked version of yesod-bin for the sole purpose of manually typing stack build yesod-bin, no? It's a little odd that it can't be marked as extra-dep: true so that it does't (for example) squelch the build output of the package we really care about... the very reason the commit was made in the first place!

wolftune added a commit to snowdriftcoop/snowdrift that referenced this issue Dec 17, 2016

extra-dep: false now needed for yesod-bin
As of stack 1.2.1 the yesod binary won't get installed in .stack-work
unless we make this change. Everything works fine as far as I can tell
with this resolution.

See commercialhaskell/stack#2849
@wolftune

This comment has been minimized.

Contributor

wolftune commented Dec 17, 2016

@mitchellwrosen no, that wasn't a reason at all (as far as I thought / know). The reason for the fork was to work with some new TH we added.

See:
https://git.snowdrift.coop/sd/yesod/commit/fa15b526dcc4acfdb4ec4536d5edadbcfcd6391d
and
https://git.snowdrift.coop/sd/yesod/commit/e6af409c68d217b8e33ef9f4e267935e07b25f65

@wolftune

This comment has been minimized.

Contributor

wolftune commented Dec 17, 2016

And, in conjunction with that, I don't find it transparent (to me at least) whether the extra-dep true or false variable would be a concern for this case (or similar cases)

@mitchellwrosen

This comment has been minimized.

Contributor

mitchellwrosen commented Dec 17, 2016

Err, I worded that wrong, but I meant you forked it to change it, "it" being an executable component and not a library.

@wolftune

This comment has been minimized.

Contributor

wolftune commented Dec 17, 2016

That sounds right… since we use yesod-bin to run essentially yesod-devel, we need it to process our code correctly. I think our separate special build script runs the install of yesod-bin perhaps. At any rate, with our setup now, nobody manually runs stack build yesod-bin (although that was tried unsuccessfully in the testing of this issue).

@mitchellwrosen

This comment has been minimized.

Contributor

mitchellwrosen commented Dec 17, 2016

And re: your last second-to-last comment, right, I think I agree, and I'm not sure that the fix was necessarily the right one (for reasons I tried to outline above but will reiterate): it's now not possible to have a forked version of an executable as part of the definition of a stack project, unless the package containing the executable is a "top-level" one. My terminology is off, but I hope I'm making sense.

@wolftune

This comment has been minimized.

Contributor

wolftune commented Dec 17, 2016

it's now not possible to have a forked version of an executable as part of the definition of a stack project…

Sounds like it could be a real problem with stack that should be at least considered and reflected upon by stack devs to figure out what to do or if this limitation is acceptable…

@wolftune

This comment has been minimized.

Contributor

wolftune commented Dec 17, 2016

@mgsloan perhaps you can now reflect on the situation enough to determine the right course of action… maybe a new issue should be opened that specifically clarifies this problem as @mitchellwrosen is describing it? I'm certainly not sure of my understanding at this point.

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Dec 19, 2016

This does look like a bug to me, mea culpa. Reopening. Does anyone have a recommendation on the best behavior here? I think the original issue I was dealing with is still a real one, but perhaps the logic needs to be closer to "only treat it as a target if it's explicitly spelled out."

@snoyberg snoyberg reopened this Dec 19, 2016

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Dec 20, 2016

OK, after a bit more research into this:

  • I was originally trying to solve #2545
  • I first wrote the patch referenced above (58d4f42), which as noted here is incorrect
  • I then immediately wrote the patch e3a0502, which properly disables test suites and benchmarks for extra-dep packages
  • It appears that with the second patch, the first patch is no longer needed, and can just be backed out

I'll open up a PR now, I'd appreciate if someone could test this use case against that PR once available.

snoyberg added a commit that referenced this issue Dec 20, 2016

@snoyberg snoyberg referenced this issue Dec 20, 2016

Merged

Local extra-dep packages can be targets #2849 #2863

2 of 2 tasks complete
@wolftune

This comment has been minimized.

Contributor

wolftune commented Dec 20, 2016

@snoyberg can you point me quickly to how to build locally from the particular git branch?

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Dec 20, 2016

$ git clone --depth=1 https://github.com/commercialhaskell/stack --branch 2849-extra-dep-target && cd stack && stack install
@wolftune

This comment has been minimized.

Contributor

wolftune commented Dec 20, 2016

right, duh. stack install is all I need, I had the branch already. I only ever did stack install x not just stack install of a repo

wolftune added a commit to snowdriftcoop/snowdrift that referenced this issue Dec 20, 2016

Revert "extra-dep: false now needed for yesod-bin"
This reverts commit 31c6588.

It turns out it was a stack bug after all, see
commercialhaskell/stack#2849
@wolftune

This comment has been minimized.

Contributor

wolftune commented Dec 20, 2016

@snoyberg I was able to build and run as expected using the stack from that PR, so confirmed fixed here

borsboom added a commit that referenced this issue Dec 20, 2016

Merge pull request #2863 from commercialhaskell/2849-extra-dep-target
Local extra-dep packages can be targets #2849
@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Dec 27, 2016

Since the fix has been confirmed, closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment