Custom Instance Domains

[hardillb]

Description

As a Team Owner,
I want to be able to supply a custom domain to be used with my projects to differentiate them from other projects hosted on the system

Customer requests

1. https://app-eu1.hubspot.com/contacts/26586079/record/0-2/8209301981
2. https://app-eu1.hubspot.com/contacts/26586079/record/0-1/7563151
3. https://app-eu1.hubspot.com/contacts/26586079/record/0-2/8763467964
4. https://app-eu1.hubspot.com/contacts/26586079/contact/8464752608

1st Iteration Development Work

Beta Give feedback

1. Custom Hostname support driver-k8s#151
   +0
   
2. Add custom hostname support to Instances #3830
   deploy:pr +1
   
3. nr-launcher authentication plugin will need updating (and possible forge app) to handle correct redirect back to the editor/dashboard if request comes from custom hostname.
4. feat: Add Custom Hostname config helm#396
   +0
   
5. Testing on dedicated scaling cluster

Production Updates

Beta Give feedback

1. Add custom domain support to pricing page under Enterprise website#2152
   +0
   
2. https://github.com/FlowFuse/CloudProject/issues/418
3. https://github.com/FlowFuse/CloudProject/issues/422

Future work

Beta Give feedback

1. Add support for custom hostname terraform-aws-flowfuse#8
   feature-request size:M - 3 +2
   
2. Support top-level domains for custom hostnames #3982
   needs-triage size:M - 3 task +3

[sammachin]

Just to confirm this should be a FQDN per project rather than a domain per team

[cgsmith]

Would this mean the FQDN would be domain.com for flowforge then each project would be a a subdomain or project folder?

project.domain.com would point to localhost:7880 or w/e?

[hardillb]

@cgsmith This is specifically talking about running with the docker or kubernetes container drivers not the localfs.

With k8s and docker drivers, the projects run in containers so do not have different port numbers on the host and are all mapped to unique hostnames on a single fixed domain at the moment. This epic is about allow teams to have their own root domain. e.g. team one can have <project name>.one.com and team two would have <project name>.two.com

You can not host projects (Node-RED instances) on different paths on a single domain (e.g. example.com/<project name>) due to the way browser local storage works as Node-RED use it as part of the editor authentication. This is a limitation of Node-RED, not the FlowForge platform.

[protocolus]

I have customers that would love to use the FF service but, depend on custom domains to serve their clients. Being able to tie a wildcard SSL to an account would be even better. That way new projects could all be under that domain automatically.

[hardillb]

First pass I see the following points that would need working on

• Support in the forge application to bind a domain to a team
• Modification to the docker/k8s container driver to allow the domain to be passed as part of project creation
• Publishing of the Ingress controller hostname to allow domain owners to setup up CNAME entries to point to the ingress controller
• Some way for the domain owner to provide/update a TLS certificate, most likely this should be a wildcard certificate

The first 3 items are relatively simple to implement, the last point will need very careful planning and may vary based on host environment. I can see a possible k8s approach, but docker may be different.

May be able to make use of letsencrypt to issue individual certs for each project.

[sammachin]

The use case of setting a domain for a team is separate to the original issue which is to set a custom FQDN for a project.

The Initial story is just per project not team domains and wildcards

[hardillb]

All the same problems still apply re TLS, the user is probably going to need to supply a certificate for the given domain. I suspect it will be easier to implement at a team level rather than per-project.

[hardillb]

@Pezmc stick a slot in the calendar and we can talk about how this will work. I think most of the work will actually be in the UI/UX apart from working out how to handle the TLS certs.

[hardillb]

Notes:

1. Scoped to a single project at a time
2. Only applies to Docker and K8s
3. Gated on a flag in team type? What is the business logic for this?
4. With out TLS it's simple
5. With TLS going to use LetsEncrypt (and work out what to do on AWS with ALB where that won't work)

Pez:

• UI/UX for entering hostname with gates based on team type - Add hostname to project settings #1344
• UI/UX about deployment of change
• Docker driver implementation (Support setting up custom hostname for project driver-docker#48)

Ben:

• k8s driver implementation (Support setting up custom hostname for project driver-k8s#66)
• Document user DNS tasks

[hardillb]

Note: Host names need to be valid and unique across all projects

[hardillb]

Blocking issue for deploying this to FlowForge Cloud https://github.com/flowforge/CloudProject/issues/110

[Pezmc]

This feature has been put on hold due to the above blocking issue.
Progress, before it was put on hold, is as follows:

• Server-side support for storing and validating hostnames: API support for setting a projects hostname #1361
• Work in progress client-side support for setting hostname: https://github.com/flowforge/flowforge/tree/wip-feat-hostname-field

When it is picked up again, the next steps were to:

• Finish the client side hostname setting and configure it to only show for EE users
• Add support for custom hostnames to the two drivers (Support setting up custom hostname for project driver-k8s#66 and Support setting up custom hostname for project driver-docker#48)
• Pass project hostnames from the server to the driver
• Document the DNS set-up process

[knolleary]

Putting this back on to the Product board for future prioritisation once https://github.com/flowforge/CloudProject/issues/110 has been planned.

[hardillb]

I have now got a second ingress-nginx controller installed in a test cluster using

controller:
  ingressClass: "custom-nginx"
  ingressClassByName: true
  ingressClassResource:
    enabled: true
    name: "custom-nginx"
    controllerValue: "flowfuse.com/ingress-nginx-2"
  config:
    proxy-body-size: "0"
    use-gzip: true
  service:
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "ssl"
      service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
      service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
      service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "120"
    externalTrafficPolicy: Local
  allowSnippetAnnotations: true

Important note, the helm release name must not match the existing nginx-ingress release (otherwise the webhook and rbac entries clash)

[joepavitt]

Once you've caught back up with this @hardillb - can you update here with a status of the outstanding items before this can be considered finished please?

[hardillb]

Current state:

• Testing of installing second nginx ingress controller on AWS EKS cluster (using the cluster built for the scaling testing)
• K8s driver Draft PR raised Custom Hostname support driver-k8s#151, the basic work is done and tested on my local setup. Will need full testing with UI
• UI and hostname storage work started in FlowFuse Project
• nr-launcher authentication plugin will need updating (and possible forge app) to handle correct redirect back to the editor/dashboard if request comes from custom hostname.

[joepavitt]

UI and hostname storage work started in FlowFuse Project

Any draft PR or linked issues for this?

[knolleary]

I've updated the description with a task list based on the latest status update.

[joepavitt]

Status update from Ben:

• UI should be functional today (PR: Add custom hostname support to Instances #3830)
• Manual testing for robustness
• Test cases to write for automated tests
• Documentation still required

Wednesday next week is when Ben hope it'll be rounded off

[hardillb]

I have this working properly in my local K8s environment.

Following the change to make the authentication redirect to the right place I will need to add a launcher version check to the UI so it's only offered on new launchers/stacks.

[hardillb]

HTTPS support appears to be working...

[joepavitt]

Can I get a status update on this please @hardillb? How are we looking for 2.5 release, or earlier?

[hardillb]

There is a large amount of extra things that will need configuring in the production cluster to enable this, it should be part of the 2.5 release, even once this is merged.

[hardillb]

I'm currently struggling to get the UI to track when the instance is restarted properly. It just gets stuck in the suspended state.

Once that is fixed then all the code should be finished.

[joepavitt]

Once coding is done, can we open relevant issues (and link back to here) to document outstanding work required to get this live please?

[hardillb]

https://github.com/FlowFuse/CloudProject/issues/418

[joepavitt]

Can this be closed off now?

[hardillb]

@joepavitt it's only because its being used as the overriding epic for the 2 bits of future work.

I can be closed for me.

[joepavitt]

Can close this once FlowFuse/terraform-aws-flowfuse#8 is resolved, #3982 will stay open as a follow-on task should the demand for it be there

[joepavitt]

Closing as majority of work is done