Enable ccache in travis

[junghans]

This is save time in build tests phase

[junghans]

@tgamblin @scheibelp

[alalazo]

I think a line here needs to be removed

The review comment didn't appear

[alalazo]

I think one line here needs to be remove. And sorry for the noise.

[alalazo]

Do you have numbers on how much time will be saved for build tests?

[junghans]

I only helps once you have a cache, but for the mpich builds in phase 3 of Travis CI it saved a min or so.

[alalazo]

😢 I hoped more, but a min is better than nothing

[alalazo]

Restarted CI.

[alalazo]

@junghans Recently I had to work on a C++ project. Bottomline, I had a much higher cache hit rate by adding:

addons:
  apt:
    sources:
      - sourceline: 'ppa:likemartinma/devel'
    packages:
      - ccache
      - ...
cache: 
  ccache: true
  ...

before_install:
   - ccache -M 2G
...

which updates ccache to the latest release, and increases the cache size. That said, I'd like to get this merged if the time for build tests is reduced noticeably 😅

[junghans]

Fine with me, ping @tgamblin @scheibelp

[junghans]

@adamjstewart all added.

[junghans]

ping @scheibelp

[scheibelp]

I'll be able to look at this by tomorrow (8/1)

[scheibelp]

Some questions:

• Enable ccache in travis #6370 (comment) adds ccache: true to the .travis.yml file, is that not necessary? (I don't see it in the diff)
• How are the spack builds making use of this if ccache is off by default in the spack config (based on ccache support #3761)?

[scheibelp]

Should this move after the next line on osx (otherwise, how is ccache found there?)

[junghans]

No, the line below it to add the compiler wrappers to the path, ccache is already in the path.

[junghans]

@scheibelp, added ccache: True.

To answer your other question, the ccache compiler wrapper are used by spack, so spack doesn't know it is ccached.

[scheibelp]

To answer your other question, the ccache compiler wrapper are used by spack, so spack doesn't know it is ccached.

Sorry but I'm not clear how that is coming about: is ccache placing a wrapper in /usr/bin/gcc (for example, or otherwise presenting itself as the compiler)?

[junghans]

It places a wrapper in /usr/lib/ccache and export PATH=/usr/lib/ccache:$PATH make spack pick up the gcc from in there instead of /usr/bin.

[scheibelp]

I think I get it now. I wish this took advantage of #3761 to get spack to use ccache explicitly. If ccache creates wrappers for all instances of gcc on PATH then the PR as is should work fine. I'd prefer not to have to reason about that though (and if something ever goes wrong with that I think we'd suddenly lose ccache benefits without knowing it).

I think that would be as simple as creating share/spack/qa/configuration/config.yaml:

config:
  ccache: true

[scheibelp]

Thanks! However: it looks like you updated .travis.yml but did not add the customized config.yaml

[junghans]

Doh!

[junghans]

@scheibelp Better now?

[scheibelp]

This seems very clear to me so thanks for that update.

However now I'm looking at sourceline: 'ppa:likemartinma/devel' - where did that come from? Is there a more-official source for this?

fixed

[alalazo]

Closed and reopened to trigger a re-build

[alalazo]

@adamjstewart I was wondering if this could be beneficial to avoid Travis erroring out when it builds mpich.

[alalazo]

@junghans Can you push any change, or rebase on develop, to see how many cache hits do we get?

[adamjstewart]

Yeah, if this helps with the mpich builds, that would be great!

[junghans]

@alalazo Done

[junghans]

@alalazo: search for ccache -s in log to see the hit rate.

[junghans]

caching seems to work again.

[alalazo]

@junghans I give up: I think I restarted the mpich build ~10 times and they always fail due to time-out (the failure obviously not being related to this PR).

[alalazo]

@junghans I am not strongly opposed to travis_wait, but I tried to avoid it because if the runner stalls for any other reason than waiting a for a long build to finish (e.g. network) we'll wait 20 mins. before failing instead of 10.

#10105 is an alternative solution that does not use travis_wait

EDIT: Travis failed with a 20 min. time-out 😭

[junghans]

@alalazo, rebased and works!

[alalazo]

Great! @tgamblin Can we merge this?

[alalazo]

@tgamblin PR merged