Support on-rio.io type DNS setup for default system-domain

[agracey]

Rio had a feature where when you installed the platform, it would set up a non-wildcard DNS entry that routes to the ingress provided.

If we did the same with Epinio, we could simplify some of the installation process. It could also mean that we generate real certs from an intermediate we control.

[manno]

Hey @agracey I converted this into a discussion, because we discuss this often and don't know how to start on it. Well and also because I wanted to see if discussions can clean up the issue list for us :)

So, if I understood correctly in todays planning, this is about using https://github.com/rancher/rdns-server
The project seems abandoned, but it basically adds the DYNDNS API protocol on top of AWS (or your own coredns).

If we use that, instead of issuing IP based hostnames, e.g. epinio.192.168.0.1.omg.howdoi.website, we could issue real names: testcluster.omg.howdoi.website.
User's could define their own names and we would create entries under our top level domain.

I'm not sure how we would generate certs for the user, that means either terminating their SSL traffic, or managing their private key? I don't see anything in rdns-server that supports this.

This story gets mixed up with the story to move away from omg.howdoi.website. I think the idea was, that we could do both in one step. However changing the omg domain to something else, seems quite easy to me. On the server side we could re-use the existing setup, just delegate another zone. On the Epinio side, we have to change 2 lines of code and docs.

Another story that also belongs in this discussion is the support for https://github.com/kubernetes-sigs/external-dns

In a broader sense, ExternalDNS allows you to control DNS records dynamically via Kubernetes resources in a DNS provider-agnostic way.

That would be useful, if users have their own DNS already for demo or production setups. However it doesn't help with the "easy" deployment. However having your own DNS, means you can configure cert-manager to do ACME DNS challenges, which allows for wildcard and private IP records with production Letsencrypt certs.

[agracey]

We do need to move away from omg.howdoi.website but also I think if we used the rdns-server there's a chance we could allow for it to switch between A and CNAME records which would mean that installation into EKS is back to a single step.

[manno]

Moving away from omg.howdoi.website seems super easy. Pick a zone, like epinio.suse.dev, delegate to @colstrom's powerdns nameserver, add another backend. Done?

Note: You'd only solve EKS for the "easy/demo" case, I'd prefer supporting external-dns.

[jandubois]

Pick a zone, like epinio.suse.dev

Just a reminder that the dev top level domain is included in the HSTS preload list of most browsers, so will not support http:// URLs (http://epinio.suse.dev would be rewritten by the browser to https://epinio.suse.dev, and then you would get a cert error). This may be a feature, but it is confusing if you are unaware.

[manno]

Interesting, should be fine, though. We also redirect http to https and do the ACME challenge over TLS.

[jimmykarily]

on-rio.io is using a deployed service (https://github.com/rancher/rdns-server) to provide that feature. The benefit of having production certs (not self signed etc) would only make sense if that service was supposed to be used in production. We can't make such guarantee, even if we decided to deploy such a service.

Both on-rio.io and omg.howdoi.website are trying to make the first experience as smooth as possible, mostly when demoing Epinio or trying things out. The omg.how.website results in a not so beautiful looking domain (has the IP address in it), but that is not necessarily bad. It acts as a reminder that it's not meant to be used in production.

The only problem I see with omg.howdoi.website is the looks of it, but that doesn't need us to use a service like on-rio.io, does it?

[manno]

on-rio.io is using a deployed service (https://github.com/rancher/rdns-server) to provide that feature

You mean it somehow provides x509 certificates? I don't see that in the code. According to their README:

The rdns-server implements the API interface of Dynamic DNS, its goal is to use a variety of DNS servers such as Route53, CoreDNS etc.

I think @agracey mentioned certicates, because we can register TXT and A records via rnds-server's DynDNS API. That means we can do Letsencrypt ACME challenges via DNS.
In contrast to the http challenge, that would allow private IP addresses and wildcards.

But I don't recognize the DynDNS API used in rdns-server. So I think it's custom? (The usual http API for DynDNS is much less RESTful I think).

Aynhow, I think neither cert-manager nor external-dns support that API. So we would have to implement:

• bypass cert-manager for the rdns-server flow
• do ACME DNS challenges via the rdns-server's DynDNS API (txt records)
• register generated or user-provided names via the rnds-server's DynDNS API (a records)

As an alternative, while checking external-dns and cert-manager, I noticed they both support DynDNS via RFC2136 (known from nsupdate, a UDP protocol I think).

However we'd still have to deal with:

• solve the TTL problem, how long does a 'lease' live, how to renew, how to tell the user?
• rogue names, so nobody registers "problematic" names, deal with over-filtering problems

[jimmykarily]

on-rio.io is using a deployed service (https://github.com/rancher/rdns-server) to provide that feature

You mean it somehow provides x509 certificates? I don't see that in the code. According to their README:

Sorry I was writing my comment before I read yours. I was replying directly to Andrew's comment (the first one). "that feature" is the dynamic DNS one.

[manno]

Oh, now it makes sense :)

[agracey]

Yes, the feature is the dynamic dns. For the certs, I was thinking we could potentially extend the project above to also allow for ACME challenges (if we wanted to).

[manno]

I think it would work already, because rdns-server supports setting TXT records via its API (which is what ACME DNS challenges use). However none of our existing tooling supports rdns-server.

Still, picking up rdns-server and making it work with Epinio is a lot.