Enabling TLS Termination at the Pod level

[cloud-j-luna]

I'm working on a feature to allow users of the Akash Network to specify that their deployments have TLS termination at the Pod level. This will allows everyone to have SSL on their deployments as long as they do the SSL Termination on the container. This is easily achievable in NGINX servers. This greatly improves decentralization as the network becomes less reliable on Cloudflare to offer encrypted traffic. Please note that providers can have access to your certificates so make sure you only use trusted providers. If you trust Cloudflare more than you trust the provider for the type of workload you will run, stay with Cloudflare

I have a working PoC and I'll submit a PR soon once I have tested everything locally. I'll also update this issue with relevant information and concerns regarding this approach.

The idea is to allow the Ingress resources to be customized to include the annotations required to perform SSL Passthrough to the Pod. The annotations are:

"nginx.ingress.kubernetes.io/backend-protocol": "HTTPS"
"nginx.ingress.kubernetes.io/ssl-passthrough": "true"

Steps to feature:

• Dynamically add annotations to the Ingress resource if SslPassthrough is enabled at the directive level.

if directive.SslPassthrough {
    result[fmt.Sprintf("%s/backend-protocol", root)] = "HTTPS"
    result[fmt.Sprintf("%s/ssl-passthrough", root)] = "true"
}

• Create the directive and add SslPassthrough according to the exposed service

directive.SslPassthrough = serviceExpose.HTTPSOptions.SslPassthrough

• Add required fields to the ManifestServiceExpose such as:

type ManifestServiceExposeHTTPSOptions struct {
    SslPassthrough bool `json:"ssl_passthrough,omitempty"`
}

• Change the CRD structure to include the fields to allow manifests to have a structure similar to this:

# ...
services:
  website:
    image: <image>
    env:
      - |
        TLS_CRT=${crt}
      - |
        TLS_KEY=${key}
    expose:
      - port: 443
        as: 80
        http_options:
          max_body_size: 104857600
        https_options:          # HERE
          ssl_passthrough: true # HERE
        to:
          - global: true
        accept:
          - ${full_domain}
# ...

I have a working version on my provider that dynamically accepts SSL Passthrough and its working perfectly (still missing the manifest part though as of writting).
Link to the fork commit: akash-network/provider@67928e0
WIP: akash-network/provider#77
What am I missing? Any suggestions?

[andy108369]

@cloud-j-luna just to clarify, this issue will include/cover the cert-manager (lets encrypt or a similar ACME compatible cert issuer) mentioned in #3 ?

[cloud-j-luna]

Hey @andy108369 , just updated the issue. And no it does not include the cert-manager yet. This one allows users to manage their own certificates.

[troian]

makes sense on changes for nginx-ingress

services:
  website:
    image: <image>
    env:
      - |
        TLS_CRT=${crt}
      - |
        TLS_KEY=${key}
    expose:
      - port: 443
        as: 80
        http_options:
          max_body_size: 104857600
        https_options:          # HERE
          ssl_passthrough: true # HERE
        to:
          - global: true
        accept:
          - ${full_domain}

SDL part tho needs some work.

1. I don't think creating a new section https_options makes sense as it creates ambivalent cases for properties like max_body_size. instead, it should be either within http_options or name the new section as options.
2. port configuration is another issue. in the example above TLS passthrough is actually done via port 80 and will not automatically redirect as this is a part of the backend service. Instead, it may look something like this:
   this example just shows configuration aspects of ports and ssl. there are other things to consider:
   • who sets redirect url
   • port 80 should not allow SSL traffic by default
   • etc

services:
  website:
    image: <image>
    env:
      - |
        TLS_CRT=${crt}
      - |
        TLS_KEY=${key}
    expose:
      - port: 8080
        as: 80
        http_options:
          max_body_size: 104857600
          redirect_https: true # default false
        to:
          - global: true
        accept:
          - ${full_domain}
      - port: 8443
        as: 443
        http_options:
          max_body_size: 104857600
        options:
          ssl_passthrough: true # default false
        to:
          - global: true
        accept:
          - ${full_domain}

[cloud-j-luna]

port configuration is another issue. in the example above TLS passthrough is actually done via port 80 and will not automatically redirect as this is a part of the backend service

Using port 443 gives users an externalPort instead of a url like port 80. The example I gave worked perfectly (with the support of @andy108369) even with SSL going through port 80 on the Ingress (forwarded to 443 on the NGINX Pod).

[arno01]

FWIW, you can leverage the IP leasing for having 443/tcp exposed directly to the pod over a leased IP (so no interference with the ingress-nginx listening). And then the tenant can put whatever x509 TLS cert he wants there (e.g. behind the nginx he'd be running) ;-)

As of:

This will allows everyone to have SSL on their deployments as long as they do the SSL Termination on the container.

This can be done by the cert-manager (Let's Encrypt) configured with the http-01 solver, so everyone can get a "green" HTTPS for their deployments. I've outlined what it takes in this comment.