HTTPS/TLS/SSL provider ingress support

[arno01]

Is your feature request related to a problem? Please describe.
People have to use 3rd party services for terminating HTTPS (TLS/SSL), i.e. CloudFlare.
Le'ts add the HTTPS support so to make Akash more decentralized! :-)

Describe the solution you'd like
There is a cert-manager for Kubernetes which supports multiple issuers, including ACME (Let's Encrypt supported!)

So it'd be cool if Akash could support that!

All that it would need is to support setting the correct annotation to the "Ingress" type of K8s resource:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt

letsencrypt is just an arbitrary name, it could be anything there.

Cluster Issuer can be configured by the Akash provider admin.
In my case I've configured it as letsencrypt:

$ kubectl get ClusterIssuers
NAME               READY   AGE
letsencrypt   True    301d

And here are the instructions on how to configure a basic ACME issuer (I am using that) => https://cert-manager.io/docs/configuration/acme/#creating-a-basic-acme-issuer
The instructions are for Staging Let's Encrypt.
So to use the Production Let's Encrypt, just change https://acme-staging-v02.api.letsencrypt.org/directory to https://acme-v02.api.letsencrypt.org/directory :-)
But it's always good to test the staging one, to make sure it is working (i.e. creating the secrets with the keys there) so to not hit the LE's rate limits.

I would see the cert manager cluster-issuer name could be configured via Akash provider's argument, in the same way we can specify the deployment runtime as of now:

$ akash provider run --help |grep runt
      --deployment-runtime-class string             kubernetes runtime class for deployments, use none for no specification (default "gvisor")

[arno01]

And then, the users may specify which services need TLS, via expose.to https://docs.akash.network/intro-to-akash/stack-definition-language#services.expose

[boz]

Thanks @arno01 - this is something we've been meaning to get to for a while but haven't had a chance to. This helps a lot.

[tmcdonough]

Would also find this useful

[cloud-j-luna]

@arno01 @boz Would a feature such as allowing the ingress controller to proxy the requests to the Pods so that we could do the TLS termination in the server such as nginx?

Something that looks like:

services:
  website:
    expose:
      - port: 80
        https_options:
          redirect: true # Proxy requests to the Pod by ignoring the TLS termination

These annotations could be used on the ingresses to achieve this.

nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"

Would such an implementation work with Akash?

Given how it is really limiting to use third-party solutions such as Cloudflare, having this as an intermediate solution, while cert-manager is not working on the Akash providers, would be a great enabler. IMO this should move higher in priority and not left behind. I would gladly work on this feature. Let me know what you think. 😄

[boz]

This would be great!

Last time I looked into this there were prohibitive limits on the number of domains that a single LE account could make certs for, but it looks like those constraints have been relaxed.

Would love to have this be available!

[andy108369]

@cloud-j-luna that's awesome, I'll be glad to test your PR. :-)
The right provider repo is here https://github.com/ovrclk/provider-services/

[andy108369]

Related #2

[cloud-j-luna]

Once this has green light we can start working on it as we also work on #2 (although they are different PRs, they are related and in the scope of our tasks/needs). 😄

[cloud-j-luna]

PRs related to cert-manager enablement:
akash-network/provider#86
akash-network/helm-charts#196

Currently only ClusterIssuer is enabled, so every deployment shares the same issuer, which by default is letsencrypt.
I would like to open the discussion on the per-deployment issuers. (Maybe on the next SIG-Providers)

CC @boz @arno01

[boz]

Love it!

This is a great milestone, we can and should look into per-deployment issuers (support custom hostnames) after this.

[andy108369]

I've enabled the TLS certs out-of-the-box in our sandbox provider, feel free to test it.

• recording https://asciinema.org/a/580716
• how-to enable that on your Akash provider https://gist.github.com/andy108369/da97a13069804004a405c342e21eb49c

[arno01]

@troian tenants can have Let's Encrypt certs out-of-the-box without the need of passing their x509 certs/keys nor tokens (since no DNS-01 ACME challenge is used, but only HTTP-01 ;)). All they need is to just point their domain to the IP or CNAME of the worker node of the cluster and make sure their SDL contains the domain of their choice under accept: field.

Please see Extra: certs for custom DNS names section in this gist to learn what it takes for the Akash to enable this feature.

Pod needs to specify cert-manager.io/cluster-issuer ingress annotation and add spec.tls bit (just the name & secretName without actual x509 secret bits at all; cert-manager will take care of that by itself once it sees the correct annotation and spec.tls bits in the deployment's ingress).

On the provider side it only requires the following:

1. installing the cert-manager;
2. configuring the HTTP-01 issuer; (basically first two steps from this gist until the DNS-01 part; unless providers wanting to have *.ingress.<provider-domain> or *.<provider-domain> signed out-of-the-box as well (the latter is currently enabled in the sandbox provider-02 and provider.akash.pro mainnet5)

Ideally, the providers should be also adding some attribute, probably tls: true to indicate the TLS support.
Possibly even the solvers too (http-01, dns-01, ...) to be complete.
The documentation will state that http-01 solver is used for the custom domains and dns-01 for the provider's ingresses (aka uri's).