CI: Cilium L4LB XDP - LoadBalancing test - Failed to connect to 10.0.0.2 port 80

[giorio94]

CI failure

Hit twice on #31013
Link: https://github.com/cilium/cilium/actions/runs/8152624430/job/22299599878

+[14:49:38] LB_VIP=10.0.0.2
++[14:49:38] docker inspect nginx -f '{{ .State.Pid }}'
+[14:49:38] nsenter -t 3500 -n /bin/sh -c 'ip a a dev eth0 10.0.0.2/32'
+[14:49:38] docker exec -t lb-node docker exec -t cilium-lb cilium-dbg service update --id 1 --frontend 10.0.0.2:80 --backends 172.18.0.3:80 --k8s-load-balancer
Deprecation warning: --id parameter will change from int to string in v1.14

Creating new service with id '1'

Added service with 1 backends

++[14:49:38] docker exec lb-node ip -o -4 a s eth0
++[14:49:38] awk '{print $4}'
++[14:49:38] cut -d/ -f1
+[14:49:39] LB_NODE_IP=172.18.0.2
+[14:49:39] ip r a 10.0.0.2/32 via 172.18.0.2
++[14:49:39] seq 1 10
+[14:49:39] for i in $(seq 1 10)
+[14:49:39] curl -o /dev/null 10.0.0.2:80
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
.....
  0     0    0     0    0     0      0      0 --:--:--  0:02:14 --:--:--     0
curl: (28) Failed to connect to 10.0.0.2 port 80 after 134353 ms: Connection timed out
+[14:51:53] echo 'Failed 1'
Failed 1
+[14:51:53] exit -1

Looking at previously failed attempts, there are a few other recent occurrences of the same error:

• https://github.com/cilium/cilium/actions/runs/8150885374/job/22277887914
• https://github.com/cilium/cilium/actions/runs/8150885374/job/22277887914
• https://github.com/cilium/cilium/actions/runs/8147376095/job/22267948615
• https://github.com/cilium/cilium/actions/runs/8146668345/job/22265671466
• https://github.com/cilium/cilium/actions/runs/8114768581/job/22185098182

[danehans]

#30921 hit this issue.
CI Job Failure: https://github.com/cilium/cilium/actions/runs/8110131035/job/22252309287

[giorio94]

Rebasing onto main seems to have helped in my case (the failure looked pretty consistent previously). So maybe the branch was missing some fixes which got in recently.

[tommyp1ckles]

@danehans did rebasing help with this as well, other two hits on this one seemed to potentially be solved with a rebase (not sure what commit however).

[danehans]

@tommyp1ckles rebasing fixed this issue for #30921.

[tommyp1ckles]

@danehans Thanks, I'm going to close for now unless someone else hits it.