HAProxy not configured correctly for server-alias with SSL

[technicalguru]

Description of the problem

I have an Ingress defined that includes an alias:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: haproxy
    ingress.kubernetes.io/hsts: "true"
    ingress.kubernetes.io/hsts-max-age: "3600"
    cert-manager.io/cluster-issuer: letsencrypt-production
    ingress.kubernetes.io/server-alias: "my-domain.com"
  name: mydomain
  namespace: myapp
spec:
  rules:
  - host: www.my-domain.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          serviceName: myservice
          servicePort: 80
  tls:
  - secretName: myapp-cert
    hosts:
    - www.my-domain.com
    - mydomain.com

But when I ask for my-domain.com then the self-signed standard certificate is being used. Checking the certificate on www.my-domain.com shows that both Common Names are correct in the certificate. The problem seems to be within the /etc/haproxy/maps/_front_bind_crt.list file that does not list the server alias:

/var/lib/haproxy/crt/mydomain_myapp-cert.pem www.my-domain.com

Expected behavior

The correct certificate is pulled by HAProxy.

Steps to reproduce the problem

1. Create an Ingress using server-alias
2. Create a certificate that contains the server name and its alias
3. Request curl -i https://www.my-domain.com => Certificate OK, webpage appears
4. Request curl -i https://my-domain.com => Certificate error shows up

Environment information

HAProxy Ingress version: v0.11

Global options:

apiVersion: v1
data:
  healthz-port: "10253"
  stats-port: "1936"
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: ingress
    meta.helm.sh/release-namespace: ingress
  creationTimestamp: "2021-01-02T09:45:57Z"
  labels:
    app.kubernetes.io/instance: ingress
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: haproxy-ingress
    app.kubernetes.io/version: v0.11
    helm.sh/chart: haproxy-ingress-0.11.0
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:data:
        .: {}
        f:healthz-port: {}
        f:stats-port: {}
      f:metadata:
        f:annotations:
          .: {}
          f:meta.helm.sh/release-name: {}
          f:meta.helm.sh/release-namespace: {}
        f:labels:
          .: {}
          f:app.kubernetes.io/instance: {}
          f:app.kubernetes.io/managed-by: {}
          f:app.kubernetes.io/name: {}
          f:app.kubernetes.io/version: {}
          f:helm.sh/chart: {}
    manager: Go-http-client
    operation: Update
    time: "2021-01-02T09:45:57Z"
  name: ingress-haproxy-ingress
  namespace: ingress
  resourceVersion: "158160442"
  selfLink: /api/v1/namespaces/ingress/configmaps/ingress-haproxy-ingress
  uid: bf5cdc1b-7f1d-4b40-be73-0d945d032bfd

Ingress objects:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: haproxy
    ingress.kubernetes.io/hsts: "true"
    ingress.kubernetes.io/hsts-max-age: "3600"
    cert-manager.io/cluster-issuer: letsencrypt-production
    ingress.kubernetes.io/server-alias: "my-domain.com"
  name: mydomain
  namespace: myapp
spec:
  rules:
  - host: www.my-domain.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          serviceName: myservice
          servicePort: 80
  tls:
  - secretName: myapp-cert
    hosts:
    - www.my-domain.com
    - mydomain.com

[technicalguru]

Just checked with v0.11.1 - same problem

[jcmoraisjr]

Hi, it seems this is another behavior of having tls rules coupled with http rules, which is the default up to v0.11 and changed in #702 to v0.12 and newer versions. Please check if possible if v0.12 fixes this - note that it's still snapshot, and read the compatibility changes before upgrading. If this fixes this issue I'll try to workaround to at least v0.11 in order to not break backward compatibility.

[technicalguru]

Tries to rollout but discovered an issue in HELM chart 0.12.0-alpha.2 (which I assume is the latest). The following error prevents haproxy to comeup:

pkg/mod/k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156: 
   Failed to watch *v1beta1.IngressClass: failed to list *v1beta1.IngressClass: ingressclasses.networking.k8s.io is forbidden: 
      User "system:serviceaccount:ingress:ingress-haproxy-ingress" cannot list resource "ingressclasses" in API group "networking.k8s.io" at the cluster scope

Checking the generated clusterrole:

# Source: haproxy-ingress/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  labels:
    helm.sh/chart: haproxy-ingress-0.12.0-alpha.2
    app.kubernetes.io/name: haproxy-ingress
    app.kubernetes.io/instance: ingress
    app.kubernetes.io/version: "v0.12-snapshot.2"
    app.kubernetes.io/managed-by: Helm
  name: ingress-haproxy-ingress
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
      - networking.k8s.io
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - extensions
      - networking.k8s.io
    resources:
      - ingresses/status
    verbs:
      - update

appears the ingressclasses is never mentioned.

Shall I open a bug or do I somth wrong?

Update - the git repo lists the class? Which HELM chart is the correct one?

[technicalguru]

Finally used the git repo for the HELM chart. The issue remains: the /etc/haproxy/maps/_front_bind_crt.list file does not list the alias or respectively all hosts mentioned in tls config

[jcmoraisjr]

Oops, missing a new release of the chart which updates the rbac config - and also the backward compatibility change in the changelog.

Thanks for the test, I'll check this shortly.

[jcmoraisjr]

Hi, I couldn't reproduce the problem in the latest snapshot, and just realized that the chart isn't updated and you didn't overwrite the controller version. Please check if using --set controller.image.tag=v0.12-snapshot.3 fixes this in your workload.

[technicalguru]

Hi @jcmoraisjr! Thank for checking. In fact, I was using v0.12-snapshot.2 for the test. Snapshot 3 fixes the issue on my cluster too. Thank you!

[jcmoraisjr]

Thanks for confirming, I'll try a non-breaking fix in the v0.11 and older.

[jcmoraisjr]

This fix comes with an incompatibility with previous versions, so it's better to leave it only in the v0.12 and newer. I'm also updating the known backward compatibility notes in the changelog. Closing.

Thanks @technicalguru for reporting and tests!

[jesuslpm]

I have the same error despite I use v0.12-snapshot.3. I have asked a question on SO. Could you please take a look? I'm really stuck with this problem.

[jcmoraisjr]

Hi, just commented in SO. you need to add the network.k8s.io group and ingressclasses resource to the rbac. I'd also strongly recommend you to create a new manifest using helm, see the getting started guide.