You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After upgrading from 7.9.1 to 7.11.0 we noticed that concourse spawned containers that didn't have any access to the internet: When you fly hijack into a container and try to curl google.com (142.251.36.206)
root@40bf25db-c326-4a7e-5899-b761145670e5:/tmp/build/get# curl 142.251.36.206
curl: (7) Failed to connect to 142.251.36.206 port 80 after 3054 ms: No route to host
Here some characteristics:
only containers seem to be offline, not workers: other containers on the same worker are online
offline containers stay offline: checked by waiting and fly hijack to test later: still offline
happens randomly: the same step might succeed in another build
happens independently of image/step that runs: Happens on all kinds of resource "check", but also in tasks
We run concourse inside a k8s (version v1.24.17-eks-4f4795d) cluster via helm chart 17.2.1
Triaging info
Concourse version: 7.11.0
Did this used to work? Yes! We never had any problems with offline containers.
The text was updated successfully, but these errors were encountered:
The next time that happens, can you see what the veth situation is on that container. You can do that by running ip addr in the container. If ip isn't installed you can usually install it via a package named iproute (or iproute2 for ubuntu/debian images).
Output will look something like this, taken from my Concourse instance:
/tmp/build/80754af9 # ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0@if917998: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1460 qdisc noqueue state UP
link/ether 9a:2e:8e:03:44:25 brd ff:ff:ff:ff:ff:ff
inet 10.254.2.50/16 brd 10.254.255.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::982e:8eff:fe03:4425/64 scope link
valid_lft forever preferred_lft forever
Summary
After upgrading from
7.9.1
to7.11.0
we noticed that concourse spawned containers that didn't have any access to the internet: When youfly hijack
into a container and try to curl google.com (142.251.36.206)Here some characteristics:
fly hijack
to test later: still offlineWe run concourse inside a k8s (version
v1.24.17-eks-4f4795d
) cluster via helm chart17.2.1
Triaging info
The text was updated successfully, but these errors were encountered: