Add KubernetesEndpointGroup

[ikhoon]

Motivation:

It is tricky to send requests to a Kubernetes cluster from outside servers without ingress.
There is no way to send traffic directly to the pod, but we can send traffic to the port of nodes (NodePort) where the pods are located.

This PR proposes a new EndpointGroup that can send requests with CSLB using NodeIP and NodePort to pods in Kubernetes. This way is not an ideal CSLB where servers and clients communicate directly, but it will be a safer way to send traffic without going through ingress which can be SPOF.

Modifications:

• Add KubernetesEndpointGroup on top of KubernetesClient to dynamically obtain Kubernetes resources.
  • Permission to watch services, nodes, pods is required to fetch endpoints.
  • service.ports[*].nodePort is used to create the port of Endpoint.
• Watch API is used to track changes in Kubernetes with a minimal delay.
  • ADDED and MODIFIED events are used to update resouces.
  • DELETED is used to remove the resouce.
  • BOOKMARK event is not used and ERROR may be ignorable.
• Test KubernetesEndpointGroup with both a real Kubernetes cluster and a mock Kubernetes server.

Result:

• You can use KubernetesEndpointGroup to perform client-side load-balancing when sending requests.
• Fixes Add K8sEndpointGroup #4497

// Create a KubernetesEndpointGroup that fetches the endpoints of the 'my-service' service in the 'default'
// namespace. The Kubernetes client will be created with the default configuration in the $HOME/.kube/config.
KubernetesClient kubernetesClient = new KubernetesClientBuilder().build();
KubernetesEndpointGroup
  .builder(kubernetesClient)
  .namespace("default")
  .serviceName("my-service")
  .build();

// If you want to use a custom configuration, you can create a KubernetesEndpointGroup as follows:
// The custom configuration would be useful when you want to access Kubernetes from outside the cluster.
Config config =
  new ConfigBuilder()
    .withMasterUrl("https://my-k8s-master")
    .withOauthToken("my-token")
    .build();
KubernetesEndpointGroup
  .builder(config)
  .namespace("my-namespace")
  .serviceName("my-service")
  .build();

[ikhoon]

I managed to finish implementing our own Kubernetes authentications by porting https://github.com/kubernetes-client/java. However, I'm not sure the code is maintainable when there are new changes on the Kubernetes side.

On second thought, it would be much better to use a well-known Kubernetes client implementation instead. https://github.com/kubernetes-client/java is hard-coded with OkHttp but https://github.com/fabric8io/kubernetes-client has an integration layer for various client implementations.

I filed fabric8io/kubernetes-client#5307 to support for Armeria backend. I will continue working on this PR once the Armeria backend for the Fabric Kubernetes client is implemented. It will make this PR simple and easy to review for maintainers.

[github-actions]

🔍 Build Scan® (commit: 6e8f01d)

Job name	Status	Build Scan®
build-windows-latest-jdk-19	✅	https://ge.armeria.dev/s/cawih3kjrkwyo
build-self-hosted-unsafe-jdk-8	✅	https://ge.armeria.dev/s/cqen6nnzz5qzs
build-self-hosted-unsafe-jdk-19-snapshot-blockhound	✅	https://ge.armeria.dev/s/hh65zgufuzix2
build-self-hosted-unsafe-jdk-17-min-java-17-coverage	✅	https://ge.armeria.dev/s/2ywgoxvttsw6a
build-self-hosted-unsafe-jdk-17-min-java-11	✅	https://ge.armeria.dev/s/vy2hyxli3v24i
build-self-hosted-unsafe-jdk-17-leak	✅	https://ge.armeria.dev/s/rotidhncnkscq
build-self-hosted-unsafe-jdk-11	✅	https://ge.armeria.dev/s/ttguy4pkr4v7q
build-macos-12-jdk-19	✅	https://ge.armeria.dev/s/lde37y67s4bly

[minwoox]

Great job! 🚀 🚀 🚀

[jrhee17]

Only nits for me 👍 Thanks @ikhoon 🙇 👍 🙇

[jrhee17]

It looks likely that either Watcher<Pod> or Watcher<Node> will emit an event.
However, both events are required to construct endpoints in maybeUpdateEndpoints.

Since most users wait for EndpointGroup#whenReady to validate the endpoint before using it, I wonder if setting allowEmptyEndpoints=false is a more sensitive default. I think as it stands, the initial endpoints will almost always be an empty list.

[ikhoon]

Agreed.

[jrhee17]

Question) Is there no need to delete the nodeToIp entry in this case since there is no valid nodeip?

[jrhee17]

Question) Does ExternalIP/HostName not work? I think it's fine to only handle InternaIP for now, but just in case someone asks.

[ikhoon]

In fact, I don't know much about how Kubernetes operates and which IP is preferred.

Let me move this filter to the builder. InternalIP will be chosen by default but overridable.

[jrhee17]

Still looks good 👍 👍 👍

[ikhoon]

There are leak reports in CI builds which turned out to be bugs in tests of https://github.com/fabric8io/kubernetes-client. I will disable the tests until the upstream fixes it.
fabric8io/kubernetes-client#5852

[ikhoon]

I found another leak in tests fabric8io/kubernetes-client#5854