Nginx Ingress Controller - Secure default backend and/or support for wildcards?

[aledbf]

From @alanhartless on April 18, 2016 23:56

Currently it's not possible to support wildcard subdomains for the Nginx ingress controller due to Kubernetes Ingress resources only allowing a legitimate domain for host. In the scenario that a wildcard domain SSL cert needs to be used for a lot of subdomains (think SaaS with custom subdomain names), it's a bit overkill to list every subdomain.

I thought of two solutions.

One - support via a flag to secure the default backend server with the default SSL certificate where the backend server can be something more than just a simple 404. This of course limits to a single cert and wouldn't be able to support multiple wildcard domains.

Two - use a flag with a "wildcard" placeholder that's replaced with * when generating the hosts. The placeholder can be a little funny looking since the host is limited to alphanumeric keys (like 0wildcard0, wildcard- or just wildcard if for sure not being used.

I've implemented both for my uses but wanted to see if either (or both) would be something interested in before I submit a PR (they feel a little hackish :-) ). Thoughts?

It would be GREAT if paths would support Nginx location regex but I know that's an Ingress limitation.

Copied from original issue: kubernetes-retired/contrib#799

[aledbf]

From @bprashanth on April 22, 2016 5:11

This is valid:
https://github.com/kelseyhightower/talks/blob/master/kubecon-eu-2016/demo/ingress/ghost.yaml#L12
You can also specify a list of hosts, and a wildcard cert, or just a list of hosts without a cert for something like SNI.

So I guess the request is that we allow wildcard matches in the host field of the ingress rules?

Two - use a flag with a "wildcard" placeholder that's replaced with * when generating the hosts

This is the specce'd behavior actually: https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/extensions/types.go#L675

@alanhartless can you give me a yaml sketch of the proposed modifcations?

[aledbf]

From @alanhartless on May 6, 2016 5:2

@bprashanth Sorry for the long delay in replying!

So I guess the request is that we allow wildcard matches in the host field of the ingress rules?

That's exactly what I'm looking for. If I could define a wildcard host in the ingress rules that would result in a nginx config similar to the following, no hack would be needed:

 server {
        listen 80;
        listen 443 ssl http2;

        server_name *.example.com;

[aledbf]

@alanhartless using server_name *.example.com; means only the first matched server (in the generated nginx.conf) will route traffic to the service, the rest is not going to be reachable. That's why you need to specify the host.

[donaldguy]

Presuming the fear is of shadowing out more specific ingresses, wouldn't it be sufficient to force all wildcard rules to the end of the server templating loop?

To this end, it seems likely that a Boolean field, or even a different key in the Rules map would be preferable to a magic string in the hostname

[donaldguy]

But I share the concern - I want my default backend to be able to serve a sane maintenance page if/when we have removed all ingress rules for a hostname (e.g. To perform a stop-the-world DB migration) and still serve the appropriate wildcard cert to not run afoul of HSTS

I may currently be able (in 0.8.3?) to set a global default cert, but that constrains me to being able to only serve sites from a single second-level domain through the ingress controller

[zarqman]

FWIW, at least in k8s 1.4.x, wildcard Ingress does work. Here's an excerpt from what we have in production:

kind: Ingress
spec:
  rules:
  - host: "*.domain.com"
    http:
      paths:
      - path: /
        backend:
          serviceName: some-app
          servicePort: 443
  tls:
  - hosts:
    - "*.domain.com"
    secretName: cert-wildcard-domain-com

In ingress-nginx 0.8.3, wildcard rules appear to be sorted to the top (it uses a straight ascii sort: * comes before _ [default 404 server usually] which comes before a-z). So, the highest sorting wildcard ingress actually becomes the default server, overriding even the "" default server. (Whether this is correct is debatable, but actually is preferable for us. nginx.tmpl could be easily enough modified to add "default_server" to the listen entry for server "" if that wasn't wanted.)

Non-wildcard entries in the generated nginx.conf still work fine (that is, they don't "shadow out" later entries). nginx pays no attention to the order of the server entries, except that in the absence of a "deafult_server" definition (as is the case in 0.8.3), it designates the first entry as the default.

So basically, most of what this issue talks about seems to already work for us in k8s 1.4 and ingress 0.8.3.

The one issue that could be helpful to address would be to able to add a "default_server" notation into the generated nginx.conf. I would suggest to add a new ConfigMap entry to define the default_server. When that entry is missing (or the related server entry cannot be found), then mark "_" as the default.

[jmn]

Checking the status on this, can this perhaps be done via annotation per ingress? So you can toggle HSTS for the wildcard domains.