Skip to content

Commit

Permalink
Configure hex, gradle, pip and npm caches.
Browse files Browse the repository at this point in the history
We saw repeated failures on some CI nodes, possibly from them being throttled
by hex.pm. To mitigate it, try to setup package caches for hex, gradle, pip and
npm.

So far it works with Docker builder only but requires that cache directories in
/home/jenkins are owned by the `jenkins` user. That has to happen as part of
the Jenkins node setup. Nodes controlled from couchdb-infra-cm have been
updated to have those directories and permission however the ARM64 one doesn't
yet, so we're excluding it temporarily.

The s390x node currently seems to have a hard time fetching hex.pm packages
specifically. All the other workers don't have that issue so it's getting
excluded as well. Linux One community issue:
linuxone-community-cloud/tickets#58
  • Loading branch information
nickva committed Oct 7, 2023
1 parent e083114 commit ede7ff2
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 16 deletions.
35 changes: 20 additions & 15 deletions build-aux/Jenkinsfile.full
Original file line number Diff line number Diff line change
Expand Up @@ -65,13 +65,16 @@ meta = [
image: "apache/couchdbci-debian:buster-erlang-${ERLANG_VERSION}"
],

'bullseye-arm64': [
name: 'Debian 11 ARM',
spidermonkey_vsn: '78',
enable_nouveau: true,
image: "apache/couchdbci-debian:bullseye-erlang-${ERLANG_VERSION}",
node_label: 'arm64v8'
],
// Exclude temporarily until we can fix package cache directory
// permissions
//
// 'bullseye-arm64': [
// name: 'Debian 11 ARM',
// spidermonkey_vsn: '78',
// enable_nouveau: true,
// image: "apache/couchdbci-debian:bullseye-erlang-${ERLANG_VERSION}",
// node_label: 'arm64v8'
// ],

'bullseye-ppc64': [
name: 'Debian 11 POWER',
Expand All @@ -81,13 +84,15 @@ meta = [
node_label: 'ppc64le'
],

'bullseye-s390x': [
name: 'Debian 11 s390x',
spidermonkey_vsn: '78',
enable_nouveau: true,
image: "apache/couchdbci-debian:bullseye-erlang-${ERLANG_VERSION}",
node_label: 's390x'
],
// Exclude until hex.pm downloads are resolved
// https://github.com/linuxone-community-cloud/tickets/issues/58
// 'bullseye-s390x': [
// name: 'Debian 11 s390x',
// spidermonkey_vsn: '78',
// enable_nouveau: true,
// image: "apache/couchdbci-debian:bullseye-erlang-${ERLANG_VERSION}",
// node_label: 's390x'
// ],

'bullseye': [
name: 'Debian 11',
Expand Down Expand Up @@ -282,7 +287,7 @@ pipeline {
// We need the jenkins user mapped inside of the image
// npm config cache below deals with /home/jenkins not mapping correctly
// inside the image
DOCKER_ARGS = '-e npm_config_cache=npm-cache -e HOME=. -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group -v /root/.gradle:/root/.gradle'
DOCKER_ARGS = '-e npm_config_cache=/home/jenkins/.npm -e HOME=. -e MIX_HOME=/home/jenkins/.mix -e HEX_HOME=/home/jenkins/.hex -e PIP_CACHE_DIR=/home/jenkins/.cache/pip -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group -v /home/jenkins/.gradle:/home/jenkins/.gradle:rw,z -v /home/jenkins/.hex:/home/jenkins/.hex:rw,z -v /home/jenkins/.npm:/home/jenkins/.npm:rw,z -v /home/jenkins/.cache/pip:/home/jenkins/.cache/pip:rw,z -v /home/jenkins/.mix:/home/jenkins/.mix:rw,z'
}

options {
Expand Down
2 changes: 1 addition & 1 deletion build-aux/Jenkinsfile.pr
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ pipeline {
// We need the jenkins user mapped inside of the image
// npm config cache below deals with /home/jenkins not mapping correctly
// inside the image
DOCKER_ARGS = '-e npm_config_cache=npm-cache -e HOME=. -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group'
DOCKER_ARGS = '-e npm_config_cache=/home/jenkins/.npm -e HOME=. -e MIX_HOME=/home/jenkins/.mix -e HEX_HOME=/home/jenkins/.hex -e PIP_CACHE_DIR=/home/jenkins/.cache/pip -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group -v /home/jenkins/.gradle:/home/jenkins/.gradle:rw,z -v /home/jenkins/.hex:/home/jenkins/.hex:rw,z -v /home/jenkins/.npm:/home/jenkins/.npm:rw,z -v /home/jenkins/.cache/pip:/home/jenkins/.cache/pip:rw,z -v /home/jenkins/.mix:/home/jenkins/.mix:rw,z'

// *** BE SURE TO ALSO CHANGE THE ERLANG VERSIONS FARTHER DOWN ***
// Search for ERLANG_VERSION
Expand Down

0 comments on commit ede7ff2

Please sign in to comment.