An 'add-source' package in sandbox always added as constraint #1362

Open
hesselink opened this Issue Jun 6, 2013 · 23 comments

Projects

None yet

6 participants

@hesselink

If you 'add-source' (and possible install) a package in a sandbox, it seems to always be added as a 'global constraint'. This can cause weird errors, and make other packages impossible to install.

For example, we have a fork of heist which I add-sourced. I later tried to build another package in the same sandbox, which requires an old snap, which needs an older heist. This made it impossible to install the package, since the new heist was always added as a global constraint.

Might be related to #1299?

@23Skidoo 23Skidoo was assigned Jun 6, 2013
@23Skidoo
Haskell member

Yes, when using a sandbox we always create an install plan for the whole sandbox, which includes constraints for all add-source deps. The idea is that if you have an add-source version of a package you usually want to build its reverse dependencies using that version. Perhaps we should change Constraint to Preference for non-modified add-source deps.

@23Skidoo 23Skidoo added a commit that referenced this issue Jun 6, 2013
@23Skidoo 23Skidoo Use soft constraints for add-source deps in sandboxInstallPolicy.
See #1362. This is an experimental change.
5940edb
@23Skidoo
Haskell member

Perhaps we should change Constraint to Preference for non-modified add-source deps.

Made that change in 5940edb. Please tell whether it works for you.

@hesselink

I just tried the latest HEAD, but it still doesn't work. Things change a bit: I can install the executable depending (transitively) on the old heist in a sandbox, and after I install a new heist in that sandbox, I can still install the executable in it.

However, once I install an (add-source'd) package depending on a new heist, I cannot install the executable depending on the old heist anymore.

@23Skidoo
Haskell member

I'll try to reproduce this.

@hesselink

I could try to minimize my case to only include public code, and attach that, would that help?

@23Skidoo
Haskell member

@hesselink

I could try to minimize my case to only include public code, and attach that, would that help?

Yes, please.

@hesselink

I've got a very simple test case here. To reproduce:

tar xfz cabal-1362.tar.gz
cd cabal-sandbox-testcase
cabal sandbox init
cabal sandbox add-source containers-fork/ package-a/ package-b/
cabal install package-b/
cabal install package-a/ # or the other way around
@23Skidoo
Haskell member

@hesselink Thanks!

@23Skidoo
Haskell member

@hesselink

It looks like this is a limitation of the constraint solver - apparently it can't handle conflicting constraints in a single install plan. This can be reproduced even without sandboxes by running cabal install containers-fork/ package-a/ package-b/ - package-a will introduce a constraint containers == 0.5.2.99 and package-b will introduce a constraint containers == 0.4.*, which will produce a conflict. Sandboxes make this issue easier to bump into because we add all installed packages to the target set - which effectively makes it impossible to have two different packages depending on different versions of the same package installed.

@kosmikus

IIUC I need --independent-goals to fix this issue - am I correct? Do have any plans to implement --independent-goals in the immediate future?

@dcoutts
Haskell member

A conversation we just had...

<refold> there is a serious limitation with the way we implemented reinstallation of revdeps for sandboxes
<dcoutts> yes?
<refold> say that we have a package-a that depends on containers == 0.4 and package-b that depends on containers == 0.5
<dcoutts> ok
<dcoutts> we should find it impossible to find a solution
<refold> then after package-a is installed into the sandbox it's impossible to install package-b
<refold> and vice versa
<dcoutts> yes
<dcoutts> that's expected, though it should be clear to users what is going on, how to change things etc
<refold> because all installed packages are added to the target set
<refold> https://github.com/haskell/cabal/issues/1362
<dcoutts> refold: yes, all packages in whole sandbox are expected to have consistent deps, by design
<dcoutts> so then the issue is UI I think, so we don't upset/confuse users
<refold> I'd say that right now it's confusing
<dcoutts> ok
<dcoutts> I can believe it
<dcoutts> presumably the solver just says something confusing
* dcoutts is not getting a response from github....
<refold> "dependency tree exhausted"
<refold> or something like that
<refold> https://status.github.com/ Major service outage.
<dcoutts> sigh
<refold> my first idea was to run the solver with --independent-goals
<dcoutts> refold: ok, suggestions?
<dcoutts> hmm
<refold> but it looks like it's not implemented
<dcoutts> refold: do you mean so that we allow it? or do you mean so we can then do an explicit check after and disallow it with a better message?
<dcoutts> I don't want to allow inconsistent sandboxes
<dcoutts> I want to move in the direction of all environments being consistent
<dcoutts> we can't do it yet for user envs, they're too big, and we don't yet have the features to manage multiple of them
<dcoutts> but we can do it now for sandboxes, they're small enough and targeted at single projects
<dcoutts> and you can have multiple of them

and interim solution...

<dcoutts> e.g. I run ghci in the sandbox and try to use both packages...
<refold> dcoutts: for now, I think, we can just print a message telling the user that sandboxes are different from a user package db if the solver fails to produce an install plan
<refold> if we're working inside a sandbox
<dcoutts> refold: mm, so we'd always give that message
<dcoutts> irrespective of the reason for failure
<dcoutts> perhaps better than nothing
<refold> dcoutts: yes
<dcoutts> refold: ok, try it
<dcoutts> hopefully temporary
<dcoutts> refold: but we should raise this with Andres and the GSoC student
<refold> dcoutts: ok, I'll talk with him
@dcoutts
Haskell member

@hesselink summary: can confirm that this behaviour is by design, but the error message is clearly not helpful, hopefully we can do something about that.

@23Skidoo
Haskell member

The error message in question:

$ cabal install package-b/
Resolving dependencies...
cabal: Could not resolve dependencies:
trying: containers-0.5.2.99/installed-aa4... (user goal)
next goal: package-b (user goal)
rejecting: package-b-0.1.0.0 (conflict: containers==0.5.2.99/installed-aa4...,
package-b => containers==0.4.*)
Dependency tree exhaustively searched.
@23Skidoo 23Skidoo added a commit that referenced this issue Jul 24, 2013
@23Skidoo 23Skidoo Improve an error message.
See #1362.
7a8d9c0
@23Skidoo
Haskell member

With my fix:

$ cabal install package-b/
Resolving dependencies...
cabal: Could not resolve dependencies:
trying: containers-0.5.2.99/installed-aa4... (user goal)
next goal: package-b (user goal)
rejecting: package-b-0.1.0.0 (conflict: containers==0.5.2.99/installed-aa4...,
package-b => containers==0.4.*)
Dependency tree exhaustively searched.
Note: when using a sandbox, all packages are required to have consistent
dependencies. Try reinstalling/unregistering the offending packages or
recreating the sandbox.
@hesselink

The error message is much better, and has helped me already. I talked with @dcoutts at Zurihac, and he seemed to think adding a flag to disable this behavior was a reasonable option. Any idea how difficult that would be?

@23Skidoo
Haskell member

@hesselink Can you open a separate ticket about this? I'll look into it later.

@hesselink

@23Skidoo I've created a separate ticket.

@tibbe
Haskell member

I might be a bit thick but I still don't understand why you want conflicting packages in a sandbox. In the end you want to build the package that the sandbox resides in, if we cannot find a coherent set of dependencies for the package, it won't build. Could someone please give a sample use case?

@hesselink

I have a use case in the separate ticket (making breaking changes in a deep dependency of an installed package). I mentioned another in this thread (having a sandbox where you build two executables depending on different versions of a dependency, which you have add-sourced). A third one is upgrading specified dependencies of your packages. You usually start at the bottom, loosening the dependency, fixing errors as they come up. Then you install, and move on to the next package. It is similar in that way to the first example.

Is any of this clear, or should I elaborate/clarify something? I keep running into this issue, but seem to have trouble making my point clear. An overarching point is perhaps that to me, a sandbox is a safe place for breaking stuff. But now if I want to break stuff, I have to do it outside the sandbox, which seems backwards.

@tibbe
Haskell member

I think I understand the issue now. You are trying to use the same sandbox for two different packages (i.e. you're using the --sandbox flag). Am I understand that part right?

@hesselink

Yes, more than two actually.

@tibbe
Haskell member

Here's how I think dependencies should work in a sandbox: we should find a consistency depedency set (or fail if we can't) for the component being built. For example, if I do

$ cabal build some-exe

that should work even if some-exe and some-other-exe don't have consistent deps. Later building some-other-exe might break some-exe. That's OK.

@hesselink

That's basically equivalent to what a 'normal' user install does, right? It sounds perfect for my use case.

@tibbe
Haskell member

@hesselink Yes, except that since we're in a sandbox it's actually safe to do. Note that a simple

$ cabal build

would still imply that we require the whole package to be consistent. Similarly you can use

$ cabal build component-1 component-2

to require those two components to be consistent.

@ezyang ezyang modified the milestone: cabal-install 2.0 Sep 6, 2016
@23Skidoo 23Skidoo removed their assignment Sep 13, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment