Health-Checks for KubernetesCRD ExternalNamedServices

[StephanSalas]

Welcome!

• Yes, I've searched similar issues on GitHub and didn't find any.
• Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What did you expect to see?

Health-Checks for KubernetesCRD ExternalNamedServices

RATIONALE: Use-Case for Hybrid Kuberenetes/Other Orchestrator Mixes

While many industry-leaders have taken the journey toward using kubernetes to deploy all things Traefik, many of us still have the need to use kubernetes as a control-plane, but not a service deployer. More specifically, there is an interesting use-case for using the KubernetesCRD provider to deploy applications to a non-kubernetes environment.

What is possible now

Anyone can use "ExternalName" serviceType with the latest version of traefik as of this writing and deploy services in a non-kubernetes environment.

What is not possible now

To add Traefik health-checks into the "ExternalName" serviceType with the latest version of Traefik.

Sources:

1. Add Healthcheck to Kubernetes CRD #6128
2. https://community.traefik.io/t/why-the-traefik-health-check-is-not-available-for-kubernetescrd-and-kubernetesingress-providers/10069

What I'd like to propose building

In the KubernetesCRD Provider, we have the simple flag "allowExternalNameServices", which allows for the aformentioned section, but the implementation of that exists within the kubernetes-native Service CRD, as it should!

Now that we've ruled that possibility out, let's make this service-crd-agnostic and have an additional healthcheck specifically within the ingressroute configuration:

routes:
  - match: Host(`example.com`) && Path(`/health`,`/v1/version`)
    kind: Rule
    services:
    - name: external-svc-with-http
      port: 9000
      healthcheck: 
        - path: /health
        interval: 5s
        timeout: 3s

This option will only work if the service is of type "externalName".

Why this is useful to the community

Two Major Reasons:

1. Not every project has the luxury, budget, or technical feasibility (think voip services, ect) for the whole project to exist within a robust virtualized orchestrator such as kubernetes, but many projects want to use kubernetes to manage dynamic configuration for key pieces of infrastructure such as traefik.

2. Having a solution to this issue would make it possible to use kubernetes as a provider for many different traefik loadbalancers in different locations, many of which may not always exist in native-kubernetes-land, but also need healthchecks in order to take advantage of traefik's endpoint-removal functionality in the event of an outage.

TLDR

Love the kuberenetesCRD functionality, and especially externalName. There are specific reasons to add a healthcheck that I've mentioned along with their value to the OS Community. Please consider my proposal/ideas and let me know if it's just a terrible idea as well :)

[StephanSalas]

@rtribotte I think I will take the initiative to build a solution and submit a PR for this issue. Because amending the design might be a bit painful as far as testing, please leave feedback if you have any as I am developing within these issue comments.

[kevinpollet]

@StephanSalas just so you know, this issue is labelled as a proposal which means that we might not accept your pull request as we need to refine the proposal and discuss more this topic.

[StephanSalas]

@kevinpollet hello thank you for the notice on the review process. I have read the contributer guidelines and will be patient as well with the project maintenance team.

Meantime, I will take a few stabs at coding within the traefik code base to gain more understanding for myself so that when we agree on a solution I can more easily get things rolling.

One consideration I missed in my original proposal was that there are actually 2 ways services are specified from an ingressroute:

1. Directly from the IngressRouteCR
2. Indirectly from the IngressRouteCR within a Traefik Service CR.

Not sure if we would want to support health checks on both from a CRD perspective, but just something I thought to start the conversation (assuming project maintainers think this is a good idea at all).

[StephanSalas]

I'm working through the solution to this currently. It seems the difficult part is the translation of objects between the provider and traefik conf[] interfaces, as well as making this backward compatible. Still requesting feedback...

[jonathon2nd]

Hello, I was wondering if this functionality is still being considered?

We have recently started developing our LB and distribution, and would like to use Traefik for this. The issue is we have some services off-k8s. Setting up externalnames and ingress/ingressroutes have not been a problem. However during testing it became quickly apparent that Traefik has no awareness when the service goes down.

We have tried some workarounds, but have nothing so far.

We were thinking we might be able to use https://doc.traefik.io/traefik/reference/dynamic-configuration/rancher/, and place labels on the service so that Traefik would do some basic health check. However, kubectl is not happy with the length of some of these labels. EDIT: Looking at https://doc.traefik.io/traefik/providers/rancher/ it seems like anything labeled 'Rancher' actually refers to old 1.x rancher, so nevermind that.

Right now the only possible workaround we have devised is to setup file provider with either a configmap or a RWX PV, less than ideal.

Has anyone else with this issue have a sufficient workaround?

[tomMoulard]

Hello @jonathon2nd,

Thanks for your interest in Traefik !

As stated here:

this issue is labelled as a proposal which means that we might not accept your pull request as we need to refine the proposal and discuss more this topic.

[tfny]

Hey y'all, so we don't really have a workaround, and a fix is not on the roadmap yet, but we would like to work with you on how to address this.

We would really love to hear your design ideas on how you would approach this.

[jonathon2nd]

We have a workaround in place that we are evaluating. We have setup an internal http workload which serves the config on a url which Traefik then gets. This way we can update the config without redeploying traefik if we use a standard configmap mounted to the workload, or a rwx pv. Not ideal but should be sufficient.

[StephanSalas]

Hi - @jonathon2nd @tfny I do have a PR that I never got to finish due to lack of time out there - #8663

Feel free to further contribute to this idea if you have more time than me.