Sandbox: "Cannot select only the dependencies" with --disable-tests

[bos]

I'm trying to use cabal 1.18 in a sandbox to build the mwc-random package. The first build succeeds, but the second one fails as follows:

+ cabal install --only-dependencies --disable-tests --enable-documentation
Resolving dependencies...
cabal: Cannot select only the dependencies (as requested by the
'--only-dependencies' flag), the package mwc-random-0.13.0.0 is required by a
dependency of one of the other targets.Note: when using a sandbox, all
packages are required to have consistent dependencies. Try
reinstalling/unregistering the offending packages or recreating the sandbox.

The problem with this error message is that it tells me that something is wrong, but it gives me no clue what is wrong, so I don't have the faintest idea what to do or where to look.

Reproduction

This takes a few minutes.

Step 1

git clone git://github.com/bos/mwc-random
cd mwc-random
cabal sandbox init
cabal install --only-dependencies -j --disable-tests
cabal install -j test-framework-hunit test-framework-quickcheck2 statistics --disable-tests
cabal configure --enable-tests
cabal build

This build should succeed without a problem.

Step 2

Now I want to do a rebuild in the same sandbox, by following the same recipe. It fails at the very first real step.

cabal install --only-dependencies --disable-tests

This is the point at which the sandbox gets stuck. I don't know what's wrong, or how to fix it.

[tibbe]

Does cabal build work (instead of cabal install) in step 2? Unlike cabal-dev, we don't use cabal install to build the main package (although this should still work).

[bos]

Step 2 is deliberately a repeat of step 1. What I'm trying to achieve is a reproducible and idempotent Jenkins build recipe, so it should work regardless of whether the workspace is empty, there are no new changes, or there are new changes (but preferably without rebuilding the entire sandbox every time).

[23Skidoo]

Interesting bug. This happens because cabal install in a sandbox tries to create an install plan for all installed packages, which adds statistics (which depends on mwc-random) to the target set, which leads to this error.

@tibbe
cabal build still works, of course.

[bos]

I kind of suspected that was what was going on, but the lack of info in the error message made it impossible to tell.

Thing is, I'm not sure what a good workaround is for now. Maybe I should try this?

cabal install --only-dependencies -j || true

[bos]

Okay, my hacky workaround "works", so I'm unblocked for now.

[23Skidoo]

Yes, the error message could definitely be improved. Since the error is caused by --only-dependencies, try running cabal install without this flag.

[tibbe]

What's the right thing for the dependency solver to do here @kosmikus ?

[23Skidoo]

@tibbe
I think it's already doing the right thing. We ask to install --only-dependencies of a set of targets, but one of the targets depends on the other target.

[23Skidoo]

@tibbe
What we need here is a better error message, which requires some integration with the solver. Or a workaround for this use case.

[tibbe]

I think it's already doing the right thing. We ask to install --only-dependencies of a set of targets, but one of the targets depends on the other target.

I guess you mean that one of the dependencies depend on the main package itself. Doesn't that mean that we have a cycle? How can we build that in the first place (without building two different copies of the main package, which I don't think works without Nix-style packages).

This all is a bit annoying. Right now we ask people to:

On first repo checkout and every time dependencies change:

cabal install -j --only-dependencies

When the package itself needs to be built:

cabal build

In fact I want the latter to imply the former so build just does the right thing. This issue means that this won't work.

My feeling is that if --only-dependencies pulls in the main package, it should be built as well. I don't want to tell people to always do

cabal install

to also just build the main package. We're not installing anything; installing the main package in the sandbox doesn't make sense; we're just trying to build.

[23Skidoo]

I guess you mean that one of the dependencies depend on the main package itself.

Yes, you're right. Only the main package is a special case and needs to be pruned from the install plan in the end (see pruneInstallPlan in D.C.Install).

My feeling is that if --only-dependencies pulls in the main package, it should be built as well.

Should it install a released version from Hackage or the current working copy?

[tibbe]

My gut feeling is that we need to use the current working copy, but I still don't understand how this works at all given the cycle that the test suite creates (as we don't consider it a separate component when we create the install plan).

[bos]

Since the error is caused by --only-dependencies, try running cabal install without this flag.

That seems to work, too. Interesting!

[23Skidoo]

@tibbe

I still don't understand how this works at all given the cycle that the test suite creates

I think it's because install is run with --disable-tests, so there's no mwc-random -> statistics -> mwc-random dependency, only statistics -> mwc-random.

[commandodev]

I've come across something that might be related. Could this be something to do with not passing cmd line args correctly:

ben@tempo:~/...|either-event-streams-local⚡ ⇒  cabal install --only-dependencies
Resolving dependencies...
All the requested packages are already installed:
Use --reinstall if you want to reinstall anyway.
ben@tempo:~/...|either-event-streams-local⚡ ⇒  cabal install --only-dependencies --reinstall
Resolving dependencies...
All the requested packages are already installed:
Use --reinstall if you want to reinstall anyway.

I'm trying to upgrade the version of pipes-concurrency in my sandbox as it's just had a bug fixed that I want to pull in. Is this even the right way to do this?

Additionally:

ben@tempo:~/...|either-event-streams-local⚡ ⇒  cabal sandbox hc-pkg unregister pipes-concurrency
ghc-pkg: unregistering pipes-concurrency would break the following packages: tempo-0.1.0.0 (use --force to override)
ben@tempo:~/...|either-event-streams-local⚡ ⇒  cabal sandbox hc-pkg unregister --force pipes-concurrency
cabal: unrecognized option `--force'
ben@tempo:~/...|either-event-streams-local⚡ ⇒  cabal sandbox hc-pkg unregister pipes-concurrency --force
cabal: unrecognized option `--force'

The argument doesn't seem to be being passed correctly to ghc-pkg (I had to give the -v flag and edit the command it runs myself).

Hope this helps - let me know it you'd prefer this as a separate bug?

[tibbe]

I believe you need to separate flags to pass to the hc-pkg command with --.

[grayjay]

#1575 covers the issue with cycles between packages and their test dependencies. I think that the only other issue here is that the error message is less clear when the build is done with --only-dependencies and a sandbox. The install command tries to reinstall all packages in the sandbox, including the test dependencies, and --only-dependencies prevents the main package from being reinstalled.

[phadej]

sandbox functionality is removed.