Network design idea: use a private network.

[rbo]

Additional to #8, it's hard to secure the public interfaces of the Hetzner servers.

Idea A: Move SDN / OpenShift traffic to a private network. Add Hetzner you can attach the dedicated server to a vSwitch.

This would end up in this network setup:

Initially, I tried this kind of setup and failed.

Because of openshift installation use interfaces with a default gateway for main interface decision. We changed the default gw to 172.22.2.1 at vlan4000 interface to force to use the VLAN interface IP.

Additional we decided to complete disable public IP because bootstrap pick the first IP and this is the public one so bootstrap etcd member uses public API but all other nodes do not have access anymore to public IPs.

Source commit #4b7523ec11161c20e4a2e851e4f2e732185e96f1

[rbo]

Moving SDN/OVN to the secondary interface is not possible, RFE: Support Migration of OVN to a Secondary Cluster Host Interface planned for 4.9

Another related RFE: Multiple NIC Support for OVN-Kubernetes Deployments

[rbo]

Idea B:

It is close to, but with an own default router

[sesheta]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[rbo]

/remove-lifecycle stale

4.9 will be released this month then we can plan to use the secondary interface thing.

[rbo]

Bring up Idea A at our internal slack with sdn engineering, and ask for a solution. https://coreos.slack.com/archives/CDCP2LA9L/p1635522685150000

/cc @durandom

[durandom]

@rbo since 4.9 is out now. Would you recommend upgrading rick to 4.9 and try this solution?
Or would we rather deploy another 4.9 test cluster and try the solution there?

[rbo]

It looks like SDN-1813 is the wrong approach, we are not the only one with that kind of problem: https://issues.redhat.com/browse/OCPBUGSM-27829

@rbo since 4.9 is out now. Would you recommend upgrading rick to 4.9 and try this solution? Or would we rather deploy another 4.9 test cluster and try the solution there?

As far as I know, the upgrade path is only available in the unsupported candidate-4.9 channel. First, we have to cleanup the rick cluster, there are some operators in "in progressing" state.

[durandom]

@larsks can you open a ticket with RH support for this and add @rbo to it?

[larsks]

@durandom I think someone else should probably manage the support ticket for this issue. It doesn't seem to touch on the MOC/BU environment, which is generally how I masquerade as a customer w/r/t support, and I'm not familiar with the hetzner environment at all. There needs to be someone else who is able to interact with the support system (or we should be pursuing this by directly interacting with the sdn team, rather than trying to treat it as a support issue -- which might be for the best because openshift support has been a mixed bag so far).

[rbo]

Mh it looks like we have an official documented solution to select the NIC for the kubelet: https://docs.openshift.com/container-platform/4.9/support/troubleshooting/troubleshooting-network-issues.html#nw-how-nw-iface-selected_troubleshooting-network-issues

[sesheta]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[rbo]

The private network works pretty well:

PR #17 contains everything. I guess we can close this issue.