Ingress + TLS using ContourGatewayProvisioner

[davinkevin]

Context:

After I found solution for 5382, I face another challenge, to make migration from Ingress to GatewayAPI with ContourGatewayProvisioner straightforward, and this is related to https.

My current configuration looks like this:

kind: GatewayClass
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
  name: legacy
spec:
  controllerName: projectcontour.io/gateway-controller
  parametersRef:
    kind: ContourDeployment
    group: projectcontour.io
    name: legacy
    namespace: projectcontour
---
kind: ContourDeployment
apiVersion: projectcontour.io/v1alpha1
metadata:
  namespace: projectcontour
  name: legacy
spec:
  contour:
    deployment:
      replicas: 1
  envoy:
    networkPublishing:
      serviceAnnotations:
        metallb.universe.tf/address-pool: legacy
---
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
  name: legacy
  namespace: projectcontour
spec:
  gatewayClassName: legacy
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      allowedRoutes:
        namespaces:
          from: All

With this, the following Ingress works well:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: httpbin
  labels:
    app: httpbin
spec:
  rules:
    - host: httpbin.192.168.74.203.nip.io
      http:
        paths:
          - backend:
              service:
                name: httpbin
                port:
                  name: http
            pathType: Prefix
            path: /

Because I have multiple Ingress setup with TLS at ingress level, using .spec.tls parameters, I would like to use them with ContourGatewayProvisioner.

And because I have dozen(s) of ingress to manage, I would like to have the simplest solution, to prevent downtime or "big bang changes" in the cluster.

NOTE I've found this solution in Gateway:

spec:
  listeners:
    - name: https
      protocol: HTTPS
      port: 443
      allowedRoutes:
        namespaces:
          from: All
      tls:
        certificateRefs:
          - name: httpbin.192.168.74.203.nip.io-469hc7bgc8
            kind: Secret

This solutions in legacy Gateway works, but it means:

• I have to duplicate every Ingress .spec.tls.* into one listener in the legacy gateway
• I have to duplicate secrets or use a Grant to allow cross-namespace secret access

What question do you have?:

• What is the official and recommended way to manage ingress with TLS using Contour Gateway Provisioner?
• Is there any Ingress full support planned for the ContourGatewayProvisioner, to avoid this kind of hacky solutions?

Anything else you would like to add:

Nothing, but please ask question if some part of this issue is not clear enough.

Environment:

• Contour version: 1.25
• Kubernetes version: (use kubectl version): 1.26
• Kubernetes installer & version: k3s-1.26
• Cloud provider or hardware configuration: Bare Metal
• OS (e.g. from /etc/os-release): Debian

[skriss]

@davinkevin to use existing Ingresses with TLS, you'll want to define your Gateway like the following:

kind: Gateway
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
  name: legacy
  namespace: projectcontour
spec:
  gatewayClassName: legacy
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      allowedRoutes:
        namespaces:
          from: All
    - name: https
      protocol: HTTP
      port: 443
      allowedRoutes:
        namespaces:
          from: All

Note that, even though the Gateway defines port 443 as protocol=HTTP, ultimately Envoy will still have TLS configured because you have those TLS details defined on the Ingress. It's not the most intuitive (and I don't think we have this currently documented), but it should work and not require you to copy all of your TLS config into Gateway Listeners for now.

This is definitely an area of ongoing development for us, so any additional details, thoughts etc. that you could provide would be valuable for the project!

[davinkevin]

👏 Fantastic!

I didn't try that because it was realy counter-intuitive, but really good to know!

From my point of view, if this is documented somewhere, I think it's a beginning. I'll try that this week end 👍 and let you know.

[skriss]

Actually, what I said above is not quite right; Contour 1.25 won't let you create HTTP listeners on two different ports. What you can use is the following:

kind: Gateway
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
  name: legacy
  namespace: projectcontour
spec:
  gatewayClassName: legacy
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      allowedRoutes:
        namespaces:
          from: All
    - name: https
      protocol: TLS
      port: 443
      tls:
        mode: Passthrough
      allowedRoutes:
        namespaces:
          from: All

You can think of the second listener as "passing through" TLS configuration to the HTTPProxy or Ingress (I know, still kind of hacky).

For 1.26 we're likely going to add support for a custom protocol for this case, e.g. projectcontour.io/https, which will mean that you don't have to specify the tls field on the Listener at all. I think this should be more obvious for users. xref. #5160 (comment)

[davinkevin]

Tested successfully in my sandbox env. The big step is now to migrate my main cluster to it 🤞.

The main point for me is to have access to documentation, with or without the custom protocol you've described.
I see more and more people investing time to use gateway api with some kind of momentum around this new API, with a better design than the Ingress. Because I follow the project for years now, I know how to do most of the things with it, but for newcomers to it, it a bit tough 😇.

Any way, thank you for your help and for this project ♥️

[skriss]

The main point for me is to have access to documentation, with or without the custom protocol you've described.
I see more and more people investing time to use gateway api with some kind of momentum around this new API, with a better design than the Ingress. Because I follow the project for years now, I know how to do most of the things with it, but for newcomers to it, it a bit tough 😇.

@davinkevin I completely agree, we need more docs here, particularly around anything unique to Contour's implementation of Gateway API. We have #4786 in the backlog for this and it's on my mind as we continue to do work here. Thanks for the feedback!