Configure hex, gradle, pip and npm caches. #4799
Conversation
nickva
force-pushed
the
jenkins-hex-cache
branch
7 times, most recently
from
a2b349c
to
40382ed
Compare
nickva
changed the title
nickva
force-pushed
the
jenkins-hex-cache
branch
2 times, most recently
from
aba7907
to
c39c9d4
Compare
|
restarted the CI and it runs through ... |
|
Thanks @big-r81! Maybe I'll try a few more rebuilds to see how it does. Eventually caches should spread across all the nodes. |
|
That sounds like a good idea.
|
|
One still failed in the same place on s390x worker....
We can check manually by logging into the workers, the ones we have access too, at least. We can see if there are any files in the cache.
Hmm we don't have anything like that. We could make separate jenkins task to do that. Like we have that jenkins docker image cleaning task. |
nickva
force-pushed
the
jenkins-hex-cache
branch
from
c39c9d4
to
5852b79
Compare
|
I also added a MIX_HOME dir as a cache option. Let's see if that makes any difference. Otherwise, I think we might want to reach out the Z Linux (s390x arch) team. It could either be a compatibility issue or somehow the CI IP range might be throttled. |
|
The caching seems to at least speed up the 'Build Release Tarball' step. I noticed on main it takes about 6 minutes and with caches it's about 5 minutes. Builds overall seem to have gotten 4-6 minutes faster. |
nickva
force-pushed
the
jenkins-hex-cache
branch
from
5852b79
to
54598f5
Compare
|
I made an issue in the Z Linux community: linuxone-community-cloud/tickets#58 For now I think we'll disable the s390x worker until that's figure out. The ARM Debian worker can also be disabled for now based on talking to the maintainer. We still have the FreeBSD arm workers to we'll be testing the architecture there, |
nickva
force-pushed
the
jenkins-hex-cache
branch
from
54598f5
to
5b665bc
Compare
We saw repeated failures on some CI nodes, possibly from them being throttled by hex.pm. To mitigate it, try to setup package caches for hex, gradle, pip and npm. So far it works with Docker builder only but requires that cache directories in /home/jenkins are owned by the `jenkins` user. That has to happen as part of the Jenkins node setup. Nodes controlled from couchdb-infra-cm have been updated to have those directories and permission however the ARM64 one doesn't yet, so we're excluding it temporarily. The s390x node currently seems to have a hard time fetching hex.pm packages specifically. All the other workers don't have that issue so it's getting excluded as well. Linux One community issue: linuxone-community-cloud/tickets#58
nickva
force-pushed
the
jenkins-hex-cache
branch
from
5b665bc
to
45eed55
Compare
Well we got 3 builds in a row. Should we merge it? |
We saw repeated failures on some CI nodes, possibly from them being throttled
by hex.pm. To mitigate it, try to setup package caches for hex, gradle, pip and
npm.
So far it works with Docker builder only but requires that cache directories in
/home/jenkins are owned by the
jenkinsuser. That has to happen as part ofthe Jenkins node setup. Nodes controlled from couchdb-infra-cm have been
updated to have those directories and permission however the ARM64 one doesn't
yet, so we're excluding it temporarily.