Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TRAVIS: enable ccache compression #2230

Closed
wants to merge 1 commit into from
Closed

Conversation

@janisozaur
Copy link
Contributor

janisozaur commented May 8, 2020

Current builds saturate Travis CI caches:

$ diff -U999 /tmp/ccache_before /tmp/ccache_after || true
--- /tmp/ccache_before	2019-11-19 21:45:52.491212479 +0000
+++ /tmp/ccache_after	2019-11-19 21:58:32.335898083 +0000
@@ -1,10 +1,13 @@
 cache directory                     /home/travis/.ccache
 primary config                      /home/travis/.ccache/ccache.conf
 secondary config      (readonly)    /etc/ccache.conf
 cache hit (direct)                     0
 cache hit (preprocessed)               0
-cache miss                             0
-no input file                          1
-files in cache                         0
-cache size                           0.0 kB
+cache miss                          4156
+called for link                       66
+called for preprocessing               3
+compile failed                         9
+no input file                          2
+files in cache                     10568
+cache size                         434.1 MB
 max cache size                     500.0 MB

I doubt it's possible to get more space allocated, though there are no
docs on what exact limits are. I suspect that default explicit setting
matches what's provided. Instead of trying to get more storage,
compress the objects with zlib (default level 6) to try increasing hit
rate.

Current builds saturate Travis CI caches:

```
$ diff -U999 /tmp/ccache_before /tmp/ccache_after || true
--- /tmp/ccache_before	2019-11-19 21:45:52.491212479 +0000
+++ /tmp/ccache_after	2019-11-19 21:58:32.335898083 +0000
@@ -1,10 +1,13 @@
 cache directory                     /home/travis/.ccache
 primary config                      /home/travis/.ccache/ccache.conf
 secondary config      (readonly)    /etc/ccache.conf
 cache hit (direct)                     0
 cache hit (preprocessed)               0
-cache miss                             0
-no input file                          1
-files in cache                         0
-cache size                           0.0 kB
+cache miss                          4156
+called for link                       66
+called for preprocessing               3
+compile failed                         9
+no input file                          2
+files in cache                     10568
+cache size                         434.1 MB
 max cache size                     500.0 MB
```

I doubt it's possible to get more space allocated, though there are no
docs on what exact limits are. I suspect that default explicit setting
matches what's provided. Instead of trying to get more storage,
compress the objects with zlib (default level 6) to try increasing hit
rate.
@janisozaur
Copy link
Contributor Author

janisozaur commented May 8, 2020

Sorry, I pasted the wrong logs.

I meant to use ones from current master: https://travis-ci.org/github/scummvm/scummvm/jobs/684756684

$ diff -U999 /tmp/ccache_before /tmp/ccache_after || true
--- /tmp/ccache_before	2020-05-08 16:34:12.987955074 +0000
+++ /tmp/ccache_after	2020-05-08 16:45:45.570927610 +0000
@@ -1,13 +1,13 @@
 cache directory                     /home/travis/.ccache
 primary config                      /home/travis/.ccache/ccache.conf
 secondary config      (readonly)    /etc/ccache.conf
 cache hit (direct)                  2816
 cache hit (preprocessed)            1045
-cache miss                       5751026
-called for link                    85473
-called for preprocessing            3832
-compile failed                     11462
-no input file                       2558
-files in cache                     11334
-cache size                         451.9 MB
+cache miss                       5755826
+called for link                    85542
+called for preprocessing            3835
+compile failed                     11471
+no input file                       2559
+files in cache                     11430
+cache size                         458.1 MB
 max cache size                     500.0 MB
@sev-
Copy link
Member

sev- commented May 12, 2020

In my opinion, ccache on travis builds has zero effect anyway: the instance gets created at each separate run, and there is no persistent storage.

Even with this, I do not understand the change. Is it intended to have a slightly bigger cache and slightly more cache misses? Is it the goal of this change? Please, explain a bit more.

@janisozaur
Copy link
Contributor Author

janisozaur commented May 12, 2020

the instance gets created at each separate run, and there is no persistent storage.

Sorry, what? Are you suggesting there were 5'755'826 (see my previous comment) cache misses for a single job? Or are you trying to contradict https://docs.travis-ci.com/user/caching/ which explicitly states:

Caches lets Travis CI store directories between builds, which is useful for storing dependencies that take longer to compile or download.

?

Is it intended to have a slightly bigger cache and slightly more cache misses?

No, the intent is to fit more cachees inside the same space and have more cache hits. As I explained in OP, current builds nearly saturate the cache and provide little improvement due to this.

That said, even if with this change the size did not decrease the way I hoped it would and I might need to append relevant entry to the config file.

@sev-
Copy link
Member

sev- commented May 12, 2020

Because I'm dumb, stupid and cannot read.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.