Can't build with --enable-tests in presence of circular dependencies #960

Closed
bos opened this Issue Jun 29, 2012 · 13 comments

Projects

None yet

10 participants

@bos
Contributor
bos commented Jun 29, 2012 edited

Update by @ezyang: when per-component build merges #3662 the builder bits are all done; all we need to do is teach the solver how to make test suite deps private, and then propagate that through the nix-local-build code.


The text library has a tests stanza that I can't use because I get an error of this form:

Preprocessing test suite 'tests' for text-0.11.2.2...
<command line>: cannot satisfy -package-id text-0.11.2.2-inplace: 
    text-0.11.2.2-inplace is shadowed by package text-0.11.2.2-fa0303a547febb0fcc06906255b11d31

I believe the problem is caused by the fact that a library that text's tests depend on uses text itself.

I realise that this is likely to be difficult to fix, but it's a problem for "core" libraries: @tibbe reports that containers faces the same difficulty.

@bos
Contributor
bos commented Jul 4, 2012

You can see another instance of this bug in mwc-random issue 15.

@kosmikus
Contributor

Yes, see #935 for the containers problem.

@kolmodin
Member
kolmodin commented Jan 2, 2014

The binary library faces the same problem.

@ttuegel ttuegel self-assigned this Sep 13, 2014
@ttuegel ttuegel added this to the Cabal-1.24 milestone Apr 23, 2015
@BardurArantsson
Collaborator

I can certainly see that this is a problem, but does anyone have any reasonable suggestions for a fix? I mean, if you have cyclical dependencies that's (in itself) a huge problem for the integrity of your tests. Should cabal(-install) try to achieve some sort of fixed-point, or what?

AFAICT this is generally an issue of a "lack of Backpack", i.e. there are really no library "interfaces" -- only implementations. As such I don't see how this can actually be (even theoretically) be solved given the current state of GHC/Cabal.

Unless we get Backpack (or similar API separation) I don't see any other real solution other than breaking apart libraries appropriately such that types and instances are separate from implementation. (But even that may not be enough.)

My pronouncement would therefore be: "Break the dependency, libraries!" (I know it's not necessarily simple.)

EDIT: ... and that this isn't something that Cabal/cabal-install can (even theoretically) solve. Thus it should be closed.

@ttuegel
Member
ttuegel commented Jun 25, 2015

@BardurArantsson The solution to this is actually very simple to state: "sandboxes always". Rather than installing the library-to-be-tested into the in-place package database, it should be installed into a sandbox. Then, its test dependencies can be installed into the sandbox. Finally, the tests can be built and run using the sandbox version of the library. This would be intellectually simple to do, but there's just a lot of things that need to be moved about to do it.

@BardurArantsson
Collaborator

Ok... let's see it! What things need to be moved about and how would you do it? :)

I understand that if the circularity is only because test-X-depends-on-impl-Y-which-depends-on-tests-for-X, it should be easy do to a sort of "tiered" compilation where the two impls are compiled first (without testing) and then tests are run afterwards. However, I didn't get the impression that that was the case from the discussion. My bad, if that was wrong. Especially since there should be no way for an impl-of-X to depend on Y-and-test-of-Y, only Y. If it's just a matter of not running tests in some circumstances, then I'm with you.

I still maintain that you should "just" separate your tests into a separate project in that case.

@ttuegel
Member
ttuegel commented Jun 26, 2015

Ok... let's see it! What things need to be moved about and how would you do it? :)

  1. Split the configure phase apart so each component can be configured separately. This will be where things are moved around, but I can't go into more detail without going and doing it.
  2. Configure tests to build against the in-place package database by default (in Cabal).
  3. Have cabal-install configure tests to build against the sandbox, rather than the in-place package database.

I still maintain that you should "just" separate your tests into a separate project in that case.

Cabal doesn't support tests-only packages, and cabal-install doesn't (yet) have features for working with multiple packages simultaneously. There are open tickets for both of these features.

@dcoutts
Member
dcoutts commented Jul 6, 2015

We think we've got a solution to this, see the last section of http://www.well-typed.com/blog/2015/07/cabal-setup-deps/

This would also lets us get rid of the fallback for when configure fails to find a solution.

@tibbe
Member
tibbe commented Jul 7, 2015

Anyone willing to implement?

@BardurArantsson
Collaborator

Cabal doesn't support tests-only packages, and cabal-install doesn't (yet) have features for working with multiple packages simultaneously. There are open tickets for both of these features.

Yes it does. You just avoid calling it a "test" project. Instead you call it a "testkit" and make it a real (normal) project. Then you depend on it in your "real" project's "test" section.

(I think the concept should be easily understood by looking a bit at the documentation for akka-testkit project.)

The general approach is more general than just this use case, but unfortunately I don't have a Haskell (demo) project to publish just ATM. (I mean, I have it, but it's not quite ready for publication.)

Does this make sense? Hopefully, I'll be able to publish my example soonish, but please look into akka-testkit in the mean time, or ask further questions! This really isn't that difficult (without extending Cabal or whatever).

@enolan enolan added a commit to enolan/cabal that referenced this issue Oct 23, 2015
@enolan enolan Test for #960
Added a test for the problem where a library package has a testsuite
that depends on a packge that depends on it. We should be able to handle
this by having separate build plans for the library and its testsuite.

The ticket for separate build plans is #1575
9993980
@23Skidoo 23Skidoo modified the milestone: Cabal 1.24, Cabal 1.26 Feb 21, 2016
@grayjay grayjay added the solver label May 30, 2016
@dcoutts
Member
dcoutts commented Jun 13, 2016

See also PR #3422

@ezyang
Contributor
ezyang commented Aug 16, 2016

When per-component build merges #3662 the builder bits are all done; all we need to do is teach the solver how to make test suite deps private, and then propagate that through the nix-local-build code.

@ezyang ezyang modified the milestone: _|_, Cabal 2.0 Aug 16, 2016
@grayjay
Collaborator
grayjay commented Nov 5, 2016

I'm closing this in favor of #1575, which has more discussion.

@grayjay grayjay closed this Nov 5, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment