K8s Ingress spec: Headless services routing for StatefulSet pods

[ivanangelov]

What would you like to be added:

K8s Ingress to be able to route external traffic to a specific pod of a StatefulSet which is exposed by a headless Service.

Example:

- host: <subdomain_1>.<ingress_host> 
  http:
    paths:
    - path: /
      backend:
        serviceName: <pod_name_1>.<headless_service_name>
        servicePort: <headless_service_port>

- host: <subdomain_2>.<ingress_host> 
  http:
    paths:
    - path: /
      backend:
        serviceName: <pod_name_2>.<headless_service_name>
        servicePort: <headless_service_port>

Why is this needed:

We are implementing a High availability setup for our K8s based solution. We are using a StatefulSet and would like to make use of the Horizontal Pod Autoscaler in order for our pods to be scaled on demand. We are using the NGINX Ingress Controller.

Furthermore, we have a logic that requires external traffic to be routed to a specific stateful pod based on the subdomain (or path) in the ingress host.

We observed that NGINX Ingress confirmed that if the above-described functionality is included in the K8s Ingress Spec they would implement it. kubernetes/ingress-nginx#3921

[ivanangelov]

/sig network

[athenabot]

/triage unresolved

Comment /remove-triage unresolved when the issue is assessed and confirmed.

🤖 I am a bot run by vllry. 👩‍🔬

[aojea]

@aledbf is not this something that is being done as part of https://github.com/kubernetes-sigs/service-apis ?

[PidgeyBE]

If you are prepared to use voyager instead of nginx as ingress controller; they seem to support this already:
https://voyagermesh.com/docs/10.0.0/guides/ingress/http/statefulset-pod/#forward-traffic-to-specific-pods-of-a-statefulset

The same API would be very useful to have for standard kubernetes/nginx ingress API indeed!

[gaurimadhok]

/assign @robscott

[robscott]

Hey @ivanangelov, thanks for filing this issue! I think we may already have added something that would solve this in Kubernetes 1.19 with the option to target any resource, not just a Service, as a backend. I've got a docs PR that covers those updates and here's a preview of what those updates will look like.

[ivanangelov]

Hello @robscott,

Thanks for the suggestion!
Actually, I need to target a backend resource of kind Pod on a specific port. Does this new feature support an explicit definition of the port of the backend resource? If so, would you provide an example?

[athenabot]

@robscott
If this issue has been triaged, please comment /remove-triage unresolved.

If you aren't able to handle this issue, consider unassigning yourself and/or adding the help-wanted label.

🤖 I am a bot run by vllry. 👩‍🔬

[ivanangelov]

Hello @robscott,

I would kindly ask you to answer my previous question so we could have some clarity about how to proceed forward with the issue.

[robscott]

Thanks for the reminder, sorry I missed your follow up question! Although the API will support referencing arbitrary resources, it will depend on implementations to define which resources they want to support targeting. If you're planning on using a specific implementation, it might be good to ask the maintainers if they have plans to support targeting additional resource types in 1.19+. With that said, API config might look like this:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-resource-backend
spec:
  defaultBackend:
    resource:
      apiGroup: "v1"
      kind: Pod
      name: example
  rules:
    - http:
        paths:
          - path: /example
            pathType: Prefix
            backend:
              resource:
                apiGroup: "v1"
                kind: Pod
                name: example

/remove-triage unresolved

[ivanangelov]

Hello @robscott,

Thanks for the example! However, what would happen if the pod example exposes multiple ports? Can we explicitly declare which port to use?
Would you direct me where I can ask the maintainers the same question?

[ivanangelov]

Hello @robscott,

I would kindly ask you to answer my previous question so we could decide how to proceed forward with the issue.

[robscott]

Hey @ivanangelov, that will depend on which Ingress controller you're using. At this point I'm not aware of any Ingress controllers that support targeting additional resource types yet. You could ask the maintainers of the controller if they have any plans for that. As an example, for ingress-nginx you might want to ask in the #ingress-nginx channel on Kubernetes Slack.

/close

[k8s-ci-robot]

@robscott: Closing this issue.

In response to this:

Hey @ivanangelov, that will depend on which Ingress controller you're using. At this point I'm not aware of any Ingress controllers that support targeting additional resource types yet. You could ask the maintainers of the controller if they have any plans for that. As an example, for ingress-nginx you might want to ask in the #ingress-nginx channel on Kubernetes Slack.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[ivanangelov]

Hello @robscott, I would like to elaborate on the last two comments. Is it possible to include in the K8s Ingres API the option of targetting specific pod + port in the rules. Because it is common for applications running in pods to expose multiple ports. I think this would be a popular use case.