Enable Travis cache

[kkm000]

Using ccache to cache object files. Also, the work is split into two jobs: one compiles everything (but skips linking binaries), and another compiles and runs tests only in libraries.

I am getting as low as 5 minutes in the first job without changes in source files, around 25 in the second. But Travis is quite inconsistent in performance: One run took almost 40 minutes, although the cache hit ratio was nominal.

Most of the changes are related to quoting the variable CXX properly.

Notable omissions:

• Compiling in single precision only. Old way was randomize on each run, but that would rub ccache's fur the wrong way. Do we need both? I'll add separate jobs for the other mode if need both.
• Not running the target ext_test. It adds a lot to compile time, partly because portaudio, but runs only 1 test in src/online2.

Closes #1805

[danpovey]

why do CC and CXX need to be quoted here?

[kkm000]

Generally, because they can be (and CXX is) a multi-token value. ccache wraps a compiler:

- CXX="ccache clang++-3.8 -Qunused-arguments -fcolor-diagnostics -Wno-tautological-compare"

[danpovey]

I don't object to anything specific in it, but @jtrmal may want to have a look; and it's still timing out.

[kkm000]

Yes, I just restarted it for the 3rd time. I did see it succeed though, and when caches warms up, timeouts will be less likely. But this is not a complete solution; tests should be able to finish with a cold cache.

I think the next step is to split testing between two or more jobs. I need to tally a successful run to understand what libs have the least interdependencies, and which of them have the longest running tests.

[kkm000]

Huh. Travis is mind-boggling. The testing job timed out thrice, then on the fourth run completed in 15 minutes (and that's with a completely cold cache!). Test run times would be more those of a high-end CPU. So much variance that its hard to even plan splitting the test job into multiple.

[jtrmal]

I think that was both mine and Dogan's experience as well. Y.

…

On Aug 14, 2017 2:07 PM, "Kirill Katsnelson" ***@***.***> wrote: Huh. Travis is mind-boggling. The testing job timed out thrice, then on the fourth run completed in 15 minutes (and that's with a completely cold cache!). Test run times would be more those of a high-end CPU. So much variance that its hard to even plan splitting the test job into multiple. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1819 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKisXwVqwtT_IfpPVNhHAHXbHFeye-LAks5sYIzmgaJpZM4O13db> .

[kkm000]

If that continues to be a problem, and alternative is e. g. Jenkins on Amazon EC. But that requires some budget and, more importanly, some work, and I can think of the way to spend time better... Let's see if Travis is still an option with all its quirks and vagaries.

[kkm000]

@jtrmal, PTAL if you have time. I am not pushing you, just pointing this is a complete change ready for a code review, and I think this is already better that what we already have, and will in any case improve the chances of a CI job running to completion. Travis pulls the master branch cache for pull requests when they do not have one on their own, then reuses this PR-specific cache if the PR branch is changed, so having a warm cache for the master branch will help.

[jtrmal]

yes, sorry, will try to review it today -- you're right, this should be dealt with asap as it improves the whole infrastructure. y.

…

On Tue, Aug 15, 2017 at 11:51 AM, Kirill Katsnelson < ***@***.***> wrote: @jtrmal <https://github.com/jtrmal>, PTAL if you have time. I am not pushing you, just pointing this is a complete change ready for a code review, and I think this is already better that what we already have, and will in any case improve the chances of a CI job running to completion. Travis pulls the master branch cache for pull requests when they do not have one on their own, then reuses this PR-specific cache if the PR branch is changed, so having a warm cache for the master branch will help. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1819 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKisX6lEVg-LPQvfc0rqCdliCCTag1Tdks5sYb6GgaJpZM4O13db> .

[jtrmal]

@danpovey LGTM, please merge.

[kkm000]

Thanks!

[danpovey]

Thanks! Merging.