[OKD4.6] ExternalIPs not working - documentation lacks of clear description

[devzeronull]

Hi,

we have a problem reaching externalIPs of a custom (non-http/s) service placed in the Hosts network 192.168.1.0/24 on UPI bare-metal installation using OVNKubernetes CNI and would appreciate a hint on how to get it work.

We have altered the cluster network configuration:
oc edit networks.config cluster

...
externalIP:
policy:
allowedCIDRs:
- 192.168.1.100/32
...

and patched a service "myservice" with an external IP:
oc patch svc myservice -n mynamespace -p '{"spec":{"externalIPs":["192.168.1.100"]}}'

The ExternalIP (192.168.1.100) of the service is reachable from another Pod within the cluster without problems but not from the outside.
We already tried to make a static route from a testing machine within the Hosts network using a master node (192.168.1.10) or a worker node (192.168.1.50) as gateway:
ip route add 192.168.1.100/32 via 192.168.1.10

Sadly the documentation doesn't say anything about how to reach external IPs it just says that it lays in the responsibility of the Networks Administrator...

Someone else was either not able to solve it which is documented in an 4 month old and still open issue in #453

So it would be very kind if anyone could point us into the right direction and tell us what is necessary to get ExternalIPs working and reachable from outside the cluster!

Best regards!

[devzeronull]

Thanks for labeling the issue - do you also have a clue on how to make ExternalIPs work?

[OleksandrShtepa]

Hi
OVNKubernetes collects IPs from interfaces on start (gateway_localnet.go: func getLocalAddrs()), no update this info latter.
In addService() it creates entry in OVN-KUBE-EXTERNALIP only if ExternalIP of Service already exists.
So, if you want to add ExternalIP you must:

1. add ExternalIP to interface on a node
2. restart OVNKubernetes on the node
3. create/modify Service

This "feature" makes ExternalIP unusable in OVNKubernetes. keepalived-operator will not works too.

[Partiks]

Yeah, I am also confused by the documentation here - https://docs.okd.io/latest/networking/configuring_ingress_cluster_traffic/configuring-externalip.html#configuring-externalip saying - "Your network infrastructure must route traffic for the external IP addresses to your cluster.". How do we do that for IPI clusters?

For UPI clusters, I had HAProxy servers sitting in front of the OKD cluster acting as proxies and I installed Keepalived on those servers and made Keepalived listen for those external IPs which made it work.

[OleksandrShtepa]

How do we do that for IPI clusters?

In case of Baremetal IPI - using keepalived-operator (works only with OpenShiftSDN) or manual by "/sbin/ip address add"

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci]

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.