cabal install; cabal test switches Cabal/GHC versions

[bergmark]

Here's a reproduction from travis: https://travis-ci.org/silkapp/json-schema/builds/27422735 (updated to a commit with a smaller diff)

The error is:

$ cabal test
cabal: You need to re-run the 'configure' command. The version of Cabal being
used has changed (was Cabal-1.16.0, now Cabal-1.18.1). Additionally the
compiler is different (was ghc-7.6, now ghc-7.4) which is probably the cause
of the problem.

This is the diff between this commit and one that builds correctly:

-  - "travis_retry cabal install --only-dependencies --force-reinstalls --enable-tests"
-  - "cabal test --jobs=1"
+  - "travis_retry cabal install --force-reinstalls --enable-tests"
+  - "cabal test"

So cabal install --only-dependencies seems to not run into this problem.

I haven't been able to reproduce this locally, but @gilligan had the same problem when using codex.

@kosmikus also mentioned that he had seen this before.

[MaxGabriel]

I had the same problem https://travis-ci.org/MaxGabriel/hackvote-yesod/builds/29276486

[ttuegel]

I think this is related to #1953; cabal test seems not to detect when it's run in a sandbox.

[gelisam]

The cause of the version switch is that the commands "configure", "build", and "test" all attempt to use the version of Cabal with which cabal-install was built, whereas the "install" command tries to use the version of Cabal which is currently installed.

Sandboxes are still indirectly involved in the problem: whereas before, cabal-install was likely to be installed to ~/.cabal along with its copy of Cabal, it is now possible to take a version of cabal-install which has been built in one sandbox and to use that executable to build a project in another sandbox.

So why isn't cabal-install always using the version of Cabal with which it was built? The reason is explained in the documentation for determineSetupMethod:

-- Decide if we're going to be able to do a direct internal call to the
-- entry point in the Cabal library or if we're going to have to compile
-- and execute an external Setup.hs script.

While I'm not sure why only "install" would need to compile and execute the external Setup.hs script (the script should be used by all four commands), it does seem plausible that cabal-install would sometimes needs to compile something with the Cabal library. I guess one solution might be to always use the version of Cabal which is installed, but that would be unfortunate, because now that people tend to keep their ~/.cabal as empty as possible, that version is likely to be very old.

[ttuegel]

With the Simple build type, the installed Cabal is never used. With the Custom build type, the cabal-install executable still needs to determine if the package has been configured before attempting later phases, so it must try to read dist/setup-config. This will be fixed when as a side effect of fixing #2214.

[gelisam]

Thanks for the quick response!

When you say that "with the Simple build type, the installed Cabal is never used", do you mean that it should never be used? Because when I use cabal-install HEAD (1.23.0.0) to compile a hello-world project which uses the Simple build type, setup-config contains the number of the system version of Cabal, not the version with which cabal-install HEAD was built.

> cabal --version
cabal-install version 1.23.0.0
using version 1.23.0.0 of the Cabal library
> rm -rf dist/ .cabal-sandbox/ cabal.sandbox.config
> cabal sandbox init
> cabal install
...
> head -n1 dist/dist-sandbox-*/setup-config
Saved package config for hint-demo-0.1.0.0 written by Cabal-1.22.1.0 using ghc-7.10

[gelisam]

Further exploration reveals that the "external build method", which uses the installed version of Cabal, is explicitly forced whenever cabal-install runs more than one job in parallel (-jN). Seems like running jobs in parallel is the default, too, because I haven't used -j in the above snippet and I can get the correct version to be written to setup-config by adding -j1:

> cabal --version
cabal-install version 1.23.0.0
using version 1.23.0.0 of the Cabal library
> rm -rf dist/ .cabal-sandbox/ cabal.sandbox.config
> cabal sandbox init
> cabal install -j1
...
> head -n1 dist/dist-sandbox-*/setup-config
Saved package config for hint-demo-0.1.0.0 written by Cabal-1.23.0.0 using ghc-7.8

Any idea why parallel builds force this external mode?

[ttuegel]

@gelisam Parallel builds happen in separate processes, not just separate threads, so I suspect that cabal-install is running the build through one of its cached Setup executables. I don't know why it is picking a different version. At least, we should be consistent with respect to in-tree builds about the version chosen.

[ndmitchell]

The closed ticket #2438 details exactly why this happens.

[ttuegel]

@gelisam Do you have Cabal-1.23.0.0 installed, or just cabal-install-1.23.0.0?

[gelisam]

Thanks Neil! I had skimmed that ticket, but its contents didn't make as much sense then as it does now.

@ttuegel Neither of them are "installed" in the sense in which I've been using the word above. I have installed cabal-install-1.23 and Cabal-1.23.0.0 in a separate sandbox, so when I run cabal-install on my hello world project, the only version of Cabal it sees installed is the one which comes with ghc 7.10.

[ndmitchell]

Cabal uses cabal.exe (the installed cabal-install binary) when doing a test, or a configure. When doing a parallel install it builds a fresh binary using the installed Cabal library and uses that. I think that compiling against the Cabal library for a Simple build is a bug, and it should be reusing the installed cabal.exe, probably making use of a hidden extra entry point.

[gelisam]

So, let's recap. The "install" command uses parallel builds, which in turn causes cabal-install to use the external setup method, which causes cabal-install to generate the source for a file called ./dist/setup/setup.hs, which must be linked against Cabal, which must be the installed somewhere, and that installed version might not be the same as the version with which cabal-install was linked.

Okay, so what is this generated setup.hs for? Must it be linked with other packages in the sandbox in which we are building, or could we somehow link it with the Cabal from the sandbox in which the current cabal-install was built?

Looking at the source of the generated setup.hs in a few sandboxes, it seems to be a copy of the user's Setup.hs, which was generated by cabal init and which few projects modify. If the user does modify this file, should all commands ("configure", "build", "install", "test") use the installed version of Cabal, or will some of them use the version with which cabal-install was linked?

[ttuegel]

We cannot link setup.hs against Cabal in the sandbox where cabal-install was built because cabal-install doesn't know about that sandbox and, in general, may not have been built in a sandbox at all! setup.hs is linked against whatever packages are in the current sandbox (or the user package database, if outside a sandbox) and the global package database.

The in-tree commands are run in the cabal-install process whenever possible, i.e. when build-type: Simple. That means these are performed using the Cabal version that cabal-install is linked against. Those commands are run in cabal-install because Cabal alone isn't smart enough to automatically re-/configure packages yet, and we consider that to be a critical feature of the user interface. That limitation will be lifted when #2214 is fixed, at which time we could consider running all commands through setup.hs. However, as Neil pointed out, there are reasons to prefer running all commands (at least for Simple packages) through cabal-install; primarily, transparency and efficiency.

[ndmitchell]

My primary goal for running commands through cabal-install is correctness. I am on a GHC which ships with a Cabal that can't tolerate spaces in the path of the Haddock binary. A Cabal bug, long since fixed. However, even though I am using a newer cabal-install, I still get a very old bug.

[gelisam]

Hmm, this discussion about build-type: Simple made me realize that the problem I am trying to help get fixed is slightly different from others in this thread. I think it might have the same cause and the same solution though, so if you don't mind, I'll describe this other symptom here instead of filing a new bug.

My build-type is not Simple. When I do use build-type: Simple, the latest version of cabal-install works like a charm, except for a strange warning about the package having never been configured. As you can see, I have slightly modified HEAD to show which version of Cabal is being used, by tracing the cabalLibVersion, and the fact that it's using a different version for the "install" and "test" commands doesn't seem to be a problem. I'm compiling in a sandbox, with a cabal-install+Cabal which I have installed in a separate sandbox.

> cabal install --enable-tests
[using Cabal-1.22.1.0]
> cabal test
Package has never been configured. Configuring with default flags. If this
fails, please run configure manually.
[using Cabal-1.23.0.0]
1 of 1 test suites (1 of 1 test cases) passed.

Now, my package is a bit unusual in that it needs to be installed before being tested, so the typical build/test/install sequence doesn't work for me. For this reason, I have used build-type: Custom to force the sequence to be build/install/test instead. Even without going into those details, on a hello-world project, if I set build-type: Custom but keep the default implementation of Setup.hs, I get the following weird behaviour in which I am unable to run the tests at all:

> cabal install --enable-tests
[using Cabal-1.22.1.0]
> cabal test
Package has never been configured. Configuring with default flags. If this
fails, please run configure manually.
[using Cabal-1.22.1.0]
cabal: You need to re-run the 'configure' command. The version of Cabal being used has changed (was Cabal-1.22.1.0, now Cabal-1.24.0.0). Additionally the compiler is different (was ghc-7.10, now ghc-7.8) which is probably the cause of the problem.
> cabal configure --enable-tests
[using Cabal-1.22.1.0]
> cabal test
cabal: You need to re-run the 'configure' command. The version of Cabal being used has changed (was Cabal-1.22.1.0, now Cabal-1.24.0.0). Additionally the compiler is different (was ghc-7.10, now ghc-7.8) which is probably the cause of the problem.    

Note that unlike what happens with #2214, running cabal configure does not resolve the problem.

So the error message I see is the same as the first message of this thread, but it seems everybody else on this thread managed to experience this with build-type: Simple, whereas I only encounter this with build-type: Custom. Still, if the fix can manage to use the same version of Cabal for all commands, regardless of the build type, then it should fix both my situation and the OP's.

[gelisam]

I just learned that cabal-install-1.22 installs a private copy of Cabal-1.22. I immediately came here hoping that it was a new feature which was added in order to fix this bug, but no, I see that we have been talking about Cabal-1.23, which must already have had this feature. So why aren't the setup files linked against that private copy? It should be the same version as the one the cabal-install executable is linked with, so all the commands should be able to use the same Cabal version.

[ttuegel]

@gelisam cabal-install-1.22 does not install a private copy of Cabal-1.22.

[gelisam]

Ah, I see, so cabal-helper-wrapper is not some internal cabal helper, it's some external tool which uses cabal. Sorry for the noise then.

[dcoutts]

The cause of the version switch is that the commands "configure", "build", and "test" all attempt to use the version of Cabal with which cabal-install was built, whereas the "install" command tries to use the version of Cabal which is currently installed.

Ok, so here's my analysis...

It's not that install "tries" to use a version of Cabal that's installed, but that when doing parallel builds we (currently) have to use multiple processes for the Setup.hs build parts, and currently that involves the "external setup method" of compiling a Setup.hs and using that for the install.

Now when it comes to build or test or whatever, we were then not in the situation where we had to force the use of the external setup method due to the multi-process issue, and so it was able to use the "internal setup method". The problem with this was twofold:

• the external one could be a different version so you get behaviour differences,
• and two it was that the selection of internal vs external was not stable. If having picked the external it stuck to always using it, things would be better.

However, now we have another fix (#2633) which allows us to use the cabal program itself as an external setup method for parallel builds. This and the new selection of setup method should fix both problems. The external and internal are now the same Cabal version. As for the stability of the choice, it's now:

determineSetupMethod options buildType'
  | buildType' == Custom             = externalSetupMethod
  | not (cabalVersion `withinRange`
         useCabalVersion options)    = externalSetupMethod
  | isJust (useLoggingHandle options)
 || forceExternalSetupMethod options = selfExecSetupMethod
  | otherwise                        = internalSetupMethod

Is this going to be stable? The build type and required cabal version are external info, coming from .cabal files and the command line. Those do switch between the external and self & internal methods, but that info itself can be considered stable. The logging and force-external conditions are internally generated choices but now these only switch between the self and internal setup methods, which are consistent with each other.

So I think I've convinced myself that this means it's stable overall. Which means I think this should be a full fix for the problem.

The other issue was about the test command not looking at the sandbox environment, which is fixed now right @ttuegel?

[ttuegel]

The other issue was about the test command not looking at the sandbox environment, which is fixed now right @ttuegel?

If you mean that cabal test builds the test suite with dependencies from the sandbox, then yes. #1953 is not fixed, but I don't think that affects this.

[tibbe]

This is fixed by #2633:

$ ~/src/cabal/cabal-install/dist/build/cabal/cabal --version
cabal-install version 1.23.0.0
using version 1.23.0.0 of the Cabal library 
$ ghc-pkg list Cabal
/usr/local/lib/ghc-7.10.1/package.conf.d
   Cabal-1.22.2.0
/Users/tibbe/.ghc/x86_64-darwin-7.10.1/package.conf.d
$ ~/src/cabal/cabal-install/dist/build/cabal/cabal install --enable-tests --dry-run -v3
...
$ ~/src/cabal/cabal-install/dist/build/cabal/cabal install --force-reinstalls --enable-tests
Resolving dependencies...
Warning: Note that reinstalls are always dangerous. Continuing anyway...
Configuring json-schema-0.7.3.6...
Building json-schema-0.7.3.6...
Installed json-schema-0.7.3.6
Updating documentation index /Users/tibbe/Library/Haskell/doc/index.html
$ ~/src/cabal/cabal-install/dist/build/cabal/cabal test --jobs=1
Preprocessing library json-schema-0.7.3.6...
...
In-place registering json-schema-0.7.3.6...
Preprocessing test suite 'json-schema-generic-aeson-tests' for
json-schema-0.7.3.6...
Running 1 test suites...
Test suite json-schema-generic-aeson-tests: RUNNING...
Test suite json-schema-generic-aeson-tests: PASS
Test suite logged to:
dist/test/json-schema-0.7.3.6-json-schema-generic-aeson-tests.log
1 of 1 test suites (1 of 1 test cases) passed.

[bergmark]

Awesome, thanks!