Add Cloudflare integration

[mms-gianni]

Add the possibility to connect to make apps publicly accessible through Cloudflare tunnels in natted environments.

Idea by BenG ( — 14.10.2023 09:58) "help-and-support"-Channel on Discord.

Proposed Operators :
https://github.com/adyanth/cloudflare-operator (My current favorite)
https://github.com/STRRL/cloudflare-tunnel-ingress-controller

[bgervan]

I have played a bit with the operator approach.

In order to reach kubero ui through a tunnel:

https://github.com/adyanth/cloudflare-operator/blob/e01ecd2dfea93bd6824169912a2ca06404d40866/docs/getting-started.md

Create api token

The tunnel setup is required to get the api key and api tokens.

Deploy Operator

kubectl apply -k https://github.com/adyanth/cloudflare-operator/config/default

Create Secrets

kubectl -n cloudflare-operator-system create secret generic cloudflare-secrets --from-literal CLOUDFLARE_API_TOKEN=apitoken --from-literal CLOUDFLARE_API_KEY=apikey

Create ClusterTunnel file

# clustertunnel.yaml
apiVersion: networking.cfargotunnel.com/v1alpha1
kind: ClusterTunnel
metadata:
  name: k3s-cluster-tunnel      # The ClusterTunnel Custom Resource Name
spec:
  newTunnel:
    name: my-k8s-tunnel         # Name of your new tunnel on Cloudflare
  size: 2                       # This is the number of replicas for cloudflared
  cloudflare:
    email: asd@asd.com     # Your email used to login to the Cloudflare Dashboard
    domain: example.com         # Domain under which the tunnel runs and adds DNS entries to
    secret: cloudflare-secrets  # The secret created before
    # accountId and accountName cannot be both empty. If both are provided, Account ID is used if valid, else falls back to Account Name.
    accountName:
    accountId: your_id_from_url
  image: cloudflare/cloudflared:2023.8.2

Apply

kubectl apply -f clustertunnel.yaml

Create TunnelBinding to kubero ui in the right namespace

---
apiVersion: networking.cfargotunnel.com/v1alpha1
kind: TunnelBinding
metadata:
  name: cloudflare-tunnel-kubero
  namespace: kubero
subjects:
  - kind: Ingress
    name: kubero
tunnelRef:
  kind: ClusterTunnel
  name: k3s-cluster-tunnel

Apply

kubectl apply -f kubero-tunnel.yaml

The UI can be reached on kubero.example.com (the default behaviour of TunnelBinding), but the fqdn can be specified as written here: https://github.com/adyanth/cloudflare-operator/blob/e01ecd2dfea93bd6824169912a2ca06404d40866/docs/configuration.md

This fqdn approach could be used for apps as well, but for a smarter solution, the right ClusterTunnel should be selected, based on the specified fqdn, when the configs are generated. Ports are configured automatically, based on the service/ingress.

[mms-gianni]

So just to sort that out:

• The Kubero UI will have a ClusterTunnel and a TunnelBinding (for the UI and the Webhooks, since there is an issue to separate them)
• Every Pipeline gets a separate ClusterTunnel (Since they mostly won't share the Domain with the Kubero UI)
• Every App gets a TunnelBinding to tunnel their Requests

1. All the ClusterTunnels CRD's are deployed along Pipelines in Kubero UI namespace and are able to share the secrets. This also solves the "Deleting a tunnel" issue, since secrets are not created with the Pipeline, but the UI.

2. The TunnelBinding is deployed in the Apps phase namespace (I assume the clustertunnel and binding don't have to be deployed in the same namespace)

It might be a problem when someone tries to create multiple Pipelines with tunnels on the same domain. But this is solvable by the UI.

Where to implement:

1. https://github.com/kubero-dev/kubero-operator/tree/main/helm-charts/kubero/templates (for the UI ClusterTunnel und TunnelBinding)
   1.1) https://github.com/kubero-dev/kubero-operator/tree/main/helm-charts/kuberopipeline (for the ClusterTunnel)

2. https://github.com/kubero-dev/kubero-operator/tree/main/helm-charts/kuberoapp/templates (for the TunnelBindings)

[bgervan]

Another note to this:
Changing the fqdn requires deleting the binding and recreate the tunnel (binding).
Ref: adyanth/cloudflare-operator#70

(I assume the clustertunnel and binding don't have to be deployed in the same namespace)
I will try that out this week.

The cloudflare tunnel limits are the following:

• In a tunnel, you can have 25 replicas, which is 50 parallel connection to cloudflare servers) - it should be configurable from the frontend, how many replicas we have.
  For this reason, the shared ClusterTunnel could be handled as the addons, with a separate menu on the sidebar (?)

[mms-gianni]

The sidebar is a topic of its own. I'm not very happy with these modal popups too, since they are too small. But Vuetify does not provide a sidebar.

Good to know the limits. They should definitely be configurable in the UI. Using the addons might be the wrong place since every CRD represents one class. So it would be required to add a TunnelBinding addon and a TunnelCluster addon.

I'd rather give it a more prominent place and place it in a separate expansion panel (comparable to resources or security). Maybe only visible in advanced mode to keep it simple by default.

[bgervan]

I meant a separate menu item, like addons, here:

And in the App edit view, a toggle for the domain to use tunnel or not, with a dropdown of your existing ClusterTunnels, similar like this:

[mms-gianni]

Why not. But if makes it a bit more complicated.

[mms-gianni]

It lives now as a standard Addon.