[FEAT] Request For Comment - Private cloud for Fluxapps

[MorningLightMountain713]

I figured before I get too far down the path working on this one, I should see what the team thinks about it.

Similar concept to AWS VPCs - A flux app can talk to every other network that the app runs on. Eg if an app is running on 3 nodes on private docker networks 172.23.5.0/24, 172.23.44.0/24 and 172.23.88.0/24 - they can all talk to each other over private VPN link.

I have a small POC running on 3 of my nodes where the private docker networks can all communicate, this is per app, so other apps can't communicate over the private link.

Would artificially limit it to something like a max of 8 instances per app with VPC.

This would be good for database clusters, where they communicate over vpn tunnels. The end user would select an option when they deploy the app to create tunnels. (and pay more) A service would then be exposed on an 169.254 address (similar to the message signing pull request I have) where they can query via http to get their networks. (or maybe expose via public api)

Uses Wireguard, the bringing up of the wireguard interface and addition of peers etc, is managed using a JS module that drives Netlink - this is a socket like interface that allows communication between userspace apps and the kernel. Can handle the adding of routes and ip addresses as well.

In order to make this work, would need to add the private docker networks to the fluxapprunning messages, so every app can know what private networks it needs to route for. While the networks are generated randomly on app deploy (I've struggled to find a way to make this deterministic, as you will always have collisions) the ip addresses of the components are deterministic (i.e, the first component will be 172.23.x.2, and the second will be 172.23.x.3 etc. So you only have to know the remote network to be able to build a table of what components are on what IP address.

Also docker networks would need to be unique per app, so a node would look in it's database for other instances, and exclude their networks before it generates a network. This would still get some collisions if two instances deploy at the same time though. However similar to what happens now, if there is a collision, the app that started latest could terminate.

While exposing the private networks gives insight into apps private addressing, it doesn't offer any attack vectors. It also doesn't have to be exposed in the public API (no harm though)

Once the apps have spun up, they listen for apprunning messages, and whenever they get one for an app they are running, they add the peer to the wireguard interface.

I understand this may add unwanted complexity or just not be a feature users want. Before implementing anything like this, I'd like to have a solid dig through the code and tidy up any tech debt and get the coverage up a bit. I see some quick wins to be had. Cheers.