New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stack build --copy-bins --test only copies binaries after testing #805

Closed
Fuuzetsu opened this Issue Aug 17, 2015 · 9 comments

Comments

Projects
None yet
4 participants
@Fuuzetsu
Contributor

Fuuzetsu commented Aug 17, 2015

Currently we can not run tests which rely on a binary produced during build. Something like

stack build && stack build --test --bench

works but rebuilds twice. Ideally stack build --test --bench would make binaries produced available before the tests run so we can use the executable for the testing.

@Fuuzetsu Fuuzetsu changed the title from stack --docker exec does not preserve PATH to stack build --copy-bins --test only copies binaries after testing Aug 17, 2015

mboes added a commit to tweag/HaskellR that referenced this issue Aug 17, 2015

mboes added a commit to tweag/HaskellR that referenced this issue Aug 17, 2015

mboes added a commit to tweag/HaskellR that referenced this issue Aug 17, 2015

@mgsloan

This comment has been minimized.

Collaborator

mgsloan commented Aug 17, 2015

I also ran into this and reported with #701, and I'm not sure what the ideal solution is.

Maybe binaries should be copied to the local install root even when tests don't pass (and do it before running tests)?

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Aug 18, 2015

Maybe binaries should be copied to the local install root even when tests don't pass (and do it before running tests)?

I did comment on that in the previous issue:

that's intentional and correct: we shouldn't be doing any installing of something which fails tests

I think the right approach is to modify the test suite to use the executable from the dist directory. You could use getExecutablePath to find the location of the test suite executable, and then ../exename/exename should give you the path to the executable just built.

There may be some issues though around not rebuilding the executable without explicitly requesting it. This is an optimization for the common case of testing a library. You may need to explicitly state that you want to build both the executable and test suite, e.g. stack build :exename :testname.

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Aug 18, 2015

IMO, the real bug here actually is that doing stack build and then stack test forces the library to be recompiled, due to the changes to cabal_macros.h. I've opened up an issue about this against cabal:

haskell/cabal#2775

With the current cabal interface, I can't think of a good way to solve this.

@Fuuzetsu

This comment has been minimized.

Contributor

Fuuzetsu commented Aug 18, 2015

The question is whether we wait for cabal or do what you suggest and look into dist.

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Aug 18, 2015

I wouldn't recommend waiting for cabal, getting a change made and released
will take much more than our usual turnaround time.

On Tue, Aug 18, 2015, 1:11 PM Mateusz Kowalczyk notifications@github.com
wrote:

The question is whether we wait for cabal or do what you suggest and look
into dist.


Reply to this email directly or view it on GitHub
#805 (comment)
.

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Aug 18, 2015

This is quickly veering off course from the original issue description, but here's my plan of attack right now (inspired by an idea @gregwebs had a few months back):

  • Check if all test suite dependencies are available
  • If they are, configure with --enable-tests

Then: cabal_macros.h won't change between stack build and stack test, which should make the compile time much smaller, hopefully obviating the speed hit described here. This will also mean that we can resolve the comment at:

Which I believe will also adversely affect you.

@snoyberg snoyberg added this to the 0.2.0.0 milestone Aug 18, 2015

@snoyberg snoyberg self-assigned this Aug 18, 2015

snoyberg added a commit that referenced this issue Aug 18, 2015

snoyberg added a commit that referenced this issue Aug 18, 2015

@mboes

This comment has been minimized.

Contributor

mboes commented Aug 18, 2015

I think the right approach is to modify the test suite to use the executable from the dist directory. You could use getExecutablePath to find the location of the test suite executable, and then ../exename/exename should give you the path to the executable just built.

That sounds like quite a bit of gymnastics for something that is a very common use case: perform system-level testing of the binary itself, e.g. to check that piping scripts through stdin and parsing all the command line arguments yields the correct result. I agree that performing say --copy-bins before --test is the wrong thing to do. But why not make the binary that was just built before the testing locally available in the PATH immediately? That is, I would expect that stack build && stack test and stack build --test to always be equivalent, yet this is not the case here.

The fact that stack build && stack test takes longer than expected is a separate bug I think. Fantastic to see that you're addressing that in #815!

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Aug 18, 2015

Actually, I didn't realize it when initially reading the issue, but I think the change I made did just that. Specifically, I think the culprit here was:

In other words: before #815, we weren't rebuilding the executable at all when doing stack test, because the reconfigure overhead from cabal was too high. #815 gets rid of that overhead with the hack described there, so now the executables are being built and copied into the local installation directory.

Can you try out the #815 PR and see how that works for you?

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Aug 23, 2015

The behavior on master for this command should now be:

  • Library and executables are built
  • Library and executables are copied to the .stack-work/install directory and library is registered
  • Test suite is built
  • Test suite is run with the install directory on the PATH
  • If everything succeeds, the executables are copied to the install path

If this is not happening, please reopen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment