Traefik return 404 in K8s when Ingress is configured, just no endpoint is healthy

[sfudeus]

Welcome!

• Yes, I've searched similar issues on GitHub and didn't find any.
• Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What did you do?

For demo purposes, I have a K8s Deployment with a single replica, an associated Service and an Ingress manifest.
The service is reachable at the defined URL and the confguration (route and service) shows up in the traefik-ui as well.

When I scale down the deployment to 0 replicas (to simulate an outage or application problem), thus having an empty list of helthy endpoints in K8s, Traefik fully removes the whole config (at least from the UI, not even the route is visible) and traefik return a 404 response when the service is accessed.

I expected in this case, that an error "503 Service Unavailable" in this case, indicating that the requested service is known, but not available currently. IIRC, Traefik1 behaved that way, but I am not sure.

What did you see instead?

I got an error 404, as if the service is not configured at all, which is different from being configured and just not available.

What version of Traefik are you using?

Version:      2.8.1
Codename:     vacherin
Go version:   go1.17.11
Built:        2022-07-11T14:06:57Z
OS/Arch:      linux/arm64

What is your environment & configuration?

containers:
  - args:
    - --global.checknewversion
    - --entrypoints.metrics.address=:9100/tcp
    - --entrypoints.traefik.address=:9000/tcp
    - --entrypoints.web.address=:8000/tcp
    - --entrypoints.websecure.address=:8443/tcp
    - --api.dashboard=true
    - --ping=true
    - --metrics.prometheus=true
    - --metrics.prometheus.entrypoint=metrics
    - --providers.kubernetescrd
    - --providers.kubernetesingress
    - --entrypoints.websecure.http.tls=true
    - --log.level=INFO
    - --providers.file.directory=/config
    - --providers.file.watch=true
    - --metrics.prometheus=true
    image: traefik:2.8.1

If applicable, please paste the log output in DEBUG level

No response

[sfudeus]

Oh, this might be related to #8802 / #8708, but this is only for IngressRoute, right? Is there an option for plain Ingress?

[sfudeus]

And I found another very old unconcluded discussion in #1077

[sfudeus]

According to #1688 and #1689 this is supposed to work for Kubernetes providers, but it seems that was lost in between, at least it seems to not work anymore for me. It seems in 2017 there was a EmptyBackendHandler middleware (from #1748) for this case, but I can't find that in the docs.

[sfudeus]

Ohmy, the docs just gave me that --providers.kubernetesingress.allowemptyservices: Allow creation of services without endpoints. (Default: false) does exist in the CLI args, too - so this might solve my issues.
Testing tomorrow. If so, sorry for the noise.

[rtribotte]

Hello @sfudeus,

Thanks for your interest in Traefik!

It seems that your issue is related to a configuration issue and the GitHub issue tracker is dedicated to bug and feature requests.

For help on your configuration, please join our Community Forum and reach out to us on the Traefik section.

We will close this issue accordingly but feel free to re-open it if you think that we missed something.