v3/networking: --network flag to attach to existing networks

[iwilltry42]

Overview

There was already some work done on using a --network flag in the master-v3 branch.
The simple goal is to be able to skip the creation of a cluster network and attach a new k3d cluster to an existing network.

Cases

1. Attaching to a pre-defined docker network (host, bridge, none)

• ✔️ here, we cannot use Aliases in the endpoint settings
  • this does not seem to much of an issue and k3d works just fine without aliases
• ℹ️ --network none is not expected to work and doesn't make to much sense to use it anyway
• ❗ --network host only supports a single-node cluster (only one master)
• 🚫 --network bridge fails to attach worker nodes to master

  time="2020-04-14T16:04:09.458596900Z" level=error msg="failed to get CA certs at https://127.0.0.1:45441/cacerts: Get https://127.0.0.1:45441/cacerts: read tcp 127.0.0.1:50948->127.0.0.1:45441: read: connection reset by peer"
  

  • Possible Solution: fetch IP from master node after creation and use this for connecting (K3S_URL) instead of the container name, as it seems like DNS doesn't work as expected here

2. Attaching to a user-defined network which is not managed by k3d

• ✔️ skip network deletion attempts when deleting the cluster
  • this is the case already, since we're not deleting the network if the cluster nodes have the label k3d.cluster.network.external=true

3. Attaching to a user-defined network which is managed by k3d (i.e. belongs to a different cluster)

• ❓ how do we handle the network once the "owner"-cluster is deleted?
  • Currently, the network will stay if there are still containers connected to it. Even, if we delete the other cluster and thus leave the network with no attached containers, it will persist.
    • Idea 1: When deleting a cluster, delete the network, if it's managed by k3d (check label) and there are no connected containers (docker will tell you), even if the network is not "owned" by the deleted cluster
    • Idea 2: leave it like it is right now and add a k3d prune or k3d cleanup command to remove unused resources

Open questions in general:

• ❓ Do we only attach the cluster to the existing network or do we create a new cluster network and attach the cluster to the existing network (this would make the cluster more "resilient" against deletion of the external network)?

Related items

• [FEATURE] Allow using existing (or default) docker bridge network, to share network between clusters #111
• [FEATURE] Support custom options for docker network #165
• [Enhancement] Allow networking between pods on k3d clusters on different physical machines #175
• [Feature] add --network flag #126
• [Feature] add --hostnetwork flag to connect cluster to host network #53

[iwilltry42]

--network flag present in v3.0.0 (See https://k3d.io/internals/networking/)