Nodeport syncing updates #45
Conversation
The original plan was to only sync NodePort service instances for the nodes they were running on. Currently, they are being synced with the pod's ip address instead, which is not guaranteed to be routable. For users running in this type of environment, this is causing these services to be reaped by the anti-entropy mechanism since their health checks fail due to the unroutability. Additionally, this provides the incorrect ip address for routing to NodePort services, which makes the sync ineffective. Given that there is not an easy way to get the node info from a service, the easiest solution to this is to sync all of the kubernetes nodes for the NodePort service. This has the benefit of distributing traffic as Kuberentes expects it for this type of service.
It *is* possible to find the node from an endpoint, so this turns into a straight up bug fix to match the intended design. It gets the routable node info into the sync, allowing folks to use it for external connections. This should also fix the anti-entropy issue that was removing NodePort services because of the nodes failing health checks.
If more than one NodePort was defined for a service, rather than taking the first, the last was taken instead. This update takes the first as expected. It additionally now takes the first port when using unnamed ports, as well as adds tests for unnamed ports for both NodePort and ClusterIP type services.
|
Hi @adilyse I found some problems while testing the nodeport-syncing branch: |
|
Thanks for taking a look! The InternalIP issue is a good find. I was working from the assumption that people were wanting to access these services from outside of their Kubernetes cluster, which would make the ExternalIP more appropriate. For your InternalIP use case, would using an InternalIP if an ExternalIP isn't found be enough? Or do you need more control? |
Some users may not be exposing any external ip addresses, or may prefer to have all the routing happen within their network. This allows users to choose whether to sync NodePort services with either the node's External or Internal IP address.
|
Thinking about this further, adding an option seems to support a wider array of use cases, so I've opted for that. |
@adilyse |
This allows for a variety of sync options with NodePort. At this time, ExternalOnly, InternalOnly and ExternalFirst are implemented options. The first two limit the node ip's address to either the ExternalIP or the InternalIP. The third option uses the node's ExternalIP if it exists. If not, it instead uses the node's InternalIP.
|
Fixes #8 |
update CHANGELOG to show connect injector RBAC support
This updates the NodePort sync logic to use the node's ip address, rather than the pod's ip. This puts a fully routable address into the sync, making it possible to externally sync with these services.
Additionally, it updates the port determination logic to take the first defined NodePort when there are multiples. It also now takes the first address when the ports are unnamed, as well as adding tests for this case.