v1.9: CI: test-upstream-k8s job fails to provision with "mkdir: cannot create directory '/home/vagrant/go/src/k8s.io': Permission denied"

[joestringer]

https://jenkins.cilium.io/job/Cilium-PR-K8s-Upstream/3247/execution/node/76/log/

(No artifacts because this is a provisioning issue)

Step "BDD-Tests"->"Start", where it runs:

cd ${TESTDIR}; vagrant ssh k8s1-${K8S_VERSION} -c "cd /home/vagrant/go/${PROJ_PATH}; ./test/kubernetes-test.sh ${DOCKER_TAG}"

Console output

15:59:25  + cd /home/jenkins/workspace/Cilium-PR-K8s-Upstream/src/github.com/cilium/cilium/test
15:59:25  + vagrant ssh k8s1-1.19 -c cd /home/vagrant/go/src/github.com/cilium/cilium; ./test/kubernetes-test.sh 398d1b9cf7810c28e5f34e959c569b1a1bd08b1d
15:59:28  serviceaccount/cilium created
15:59:28  serviceaccount/cilium-operator created
15:59:28  configmap/cilium-config created
15:59:28  clusterrole.rbac.authorization.k8s.io/cilium created
15:59:28  clusterrole.rbac.authorization.k8s.io/cilium-operator created
15:59:28  clusterrolebinding.rbac.authorization.k8s.io/cilium created
15:59:28  clusterrolebinding.rbac.authorization.k8s.io/cilium-operator created
15:59:28  daemonset.apps/cilium created
15:59:28  deployment.apps/cilium-operator created
15:59:28  Running pods 0
15:59:29  Running pods 0
15:59:31  Running pods 0
15:59:32  Running pods 0
15:59:33  Running pods 0
15:59:34  Running pods 0
15:59:35  Running pods 0
15:59:37  Running pods 0
15:59:38  Running pods 1
15:59:39  Running pods 1
15:59:40  Running pods 1
15:59:41  Running pods 1
15:59:42  Running pods 1
15:59:43  Running pods 1
15:59:45  Running pods 1
15:59:46  Running pods 1
15:59:46  Running pods 1
15:59:48  Running pods 1
15:59:49  Running pods 2
15:59:49  result match, continue with kubernetes
15:59:49  Installing kubetest manually
15:59:49  mkdir: cannot create directory '/home/vagrant/go/src/k8s.io': Permission denied

[joestringer]

Looks like this was specifically caused by backporting #16589 to v1.9, must be a regression somehow between the VM image and the v1.9 tree.

[pchaigno]

Likely caused by cilium/packer-ci-build#262. /cc @twpayne

It works on master, so maybe the test doesn't use that directory or does something different on master.

[pchaigno]

git diff origin/v1.9..origin/master test/kubernetes-test.sh has nothing of interest but git diff origin/v1.9..origin/master jenkinsfiles/kubernetes-upstream.Jenkinsfile has the following change, which likely explains why it doesn't fail on master.

@@ -94,7 +87,7 @@ pipeline {
             }
 
             steps {
-                sh 'cd ${TESTDIR}; vagrant ssh k8s1-${K8S_VERSION} -c "cd /home/vagrant/go/${PROJ_PATH}; ./test/kubernetes-test.sh ${DOCKER_TAG}"'
+                sh 'cd ${TESTDIR}; vagrant ssh k8s1-${K8S_VERSION} -c "cd /home/vagrant/go/${PROJ_PATH}; sudo ./test/kubernetes-test.sh ${DOCKER_TAG}"'
             }
         }

That change was introduced in #15397. I'll mark for backports.

[github-actions]

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.