travis: All GHCs use same cache; check whether that's OK

[petertseng]

Currently, our Travis builds use the same package cache for all three versions of GHC that we test, which could cause strange behaviors (but luckily has not, so far).

This bit me when I was setting up Travis on one of my own projects. This will not necessarily be an problem for this track (and this issue could be closed if we think that), but this issue will serve as documentation of what could happen and how to fix it.

If there are problems related to this, I predict symptoms will manifest in one of two ways:

1. (If travis: Automatically invalidate caches if new package versions exist #129 gets implemented) Cache misses even when no package versions have changed.
   This happened to me because a package was only being pulled in as a dependency in certain GHC versions. The inconsistency caused my different versions to fight with each other for which list of packages was correct to cache (It means I never got cache hits even when I should have).
2. Failure to find a package that we supposedly installed, something like:

test.hs:1:8:
    Could not find module ‘Package.We.Definitely.Have’
    Use -v to see a list of the files searched for.

If this becomes a problem, the remediation is to add a compiler: key in the matrix in travis.yml (https://github.com/hvr/multi-ghc-travis/blob/master/make_travis_yml.hs will generate an example yaml file, or take a look at https://github.com/ekmett/lens/blob/master/.travis.yml for a generated example). This will cause Travis to use separate caches for each GHC version, which should avoid both issues above.

You can confirm the cache behavior by looking at the build logs and expand the arrow next to "Setting up build cache". You should see a line that looks like:

attempting to download cache archive
fetching master/cache--compiler-gcc.tgz
found cache

If all builds are using the same cache, that tgz name will be the same across all versions. If they are different, you might instead see something like:

attempting to download cache archive
fetching master/cache--compiler-GHC7.6.3.tgz
found cache

[petertseng]

In general, it certainly seems that how Haskell is currently set up on Travis (see #127 for another investigation on the state of the world) kind of gives the feeling of "held together with duct tape". The external repository needed to get 7.10, the hack using the compiler: to separate caches... it's fascinating in a way.

[rbasso]

After we merge #128 , the distinct env: GCHVER=X.Y.Z in the build matrix will force Travis to use multiples caches, right?

I propose we wait that to check if the cache problem goes away..

[petertseng]

https://github.com/travis-ci/travis-build/blob/0cd467fff60472c85c9e81735ddeefebd1448b0d/lib/travis/build/script/c.rb#L33 / https://github.com/travis-ci/travis-build/blob/maser/lib/travis/build/script/c.rb#L33 seem to indicate that the cache used depends solely on the compiler:, and does not read the environment variable. At least that was the result of my experimentation two months ago - I have not experimented further since then, so circumstances may have changed.

In either case, it's agreed we can wait. Another reason we can wait is I haven't observed cache problems with this particular project just yet.

[rbasso]

I checked Travis build 315's raw logs and looks like it is respecting the build matrix and using distinct cache files as expected.

#315.1 GHCVER=7.6.3  cache-linux-precise-7dfa9d634301582ab17bc64a1543d2889a34379649fe26c39747a9e3f6bd5747--compiler-gcc.tgz
#315.2 GHCVER=7.8.4  cache-linux-precise-4578989966debb5e1a6860ef61caa638c61013510eb6b8304b6406519a116e61--compiler-gcc.tgz
#315.3 GHCVER=7.10.3 cache-linux-precise-e8efcbc960501c99f8d1b51be1c7b5d044576c697ea3846092a0794121a13487--compiler-gcc.tgz
#315.4 GHCVER=8.0.1  cache-linux-precise-2d245a700824c324da504a641cb1bd34288f20b1e9d77ca3a5f8de8f5d82224b--compiler-gcc.tgz

In build 313.4, I guess it didn't find the cache, because it was the first build with GHC 8.0.1 in master:

attempting to download cache archive
fetching master/cache-linux-precise-2d245a700824c324da504a641cb1bd34288f20b1e9d77ca3a5f8de8f5d82224b--compiler-gcc.tgz
fetching master/cache--compiler-gcc.tgz
could not download cache

I guess the cache problem is gone, right?

Am I missing something here, @petertseng ?

And thank you for all the PRs related to _Travis_. 👍

[petertseng]

Excellent, since the presence of travis-ci/travis-build@89b5d3f#diff-2c4750a8306b986377ec52a74df7a87f , the problem I described is solved automatically and the compiler: key is no longer the only thing being taken into account - the caches are being split by the ENV var contents as well! So the remediation I described is no longer necessary!