[Proposal] Refactor network config from cluster spec into a New Network Spec

[geojaz]

This was discussed at kops office hours on 12/22 and I believe folks have been thinking about this idea for some time.

I propose that it's time to pull network config (VPC, subnets, routes, etc) out of the cluster spec and into a first class networking spec/object. We currently have the ability to configure networking in the cluster spec, but as kops networking moves away from the aws vision of the world and morphs into a more agnostic cloud/bare metal networking, I think a refactor like this would let us address more varied configs in a more manageable way.

First steps are probably to pull the existing descriptors out from the cluster spec and move them into a new spec (which will be called Network). Next step is making this spec general enough to address the needs of the community including different types of egress, routes, cidrs and many others that I'm leaving out.

I'd like to get comments from the community about pain points for network management and how you would be interested in using kops to manage those- especially associated with the cluster that kops creates/manages. I'm also interested to hear if this seems like a good use of time or not. We could certainly continue to manage the network from within the cluster spec- but

My specific interest in this relates to adding routes to subnets that are created/managed by kops. Currently, I manage them via TF, but this is kind of clunky.

[geojaz]

@justinsb @chrislovecnm

[chrislovecnm]

Sounds great, but this will be a fun migration. We probably need some tool for this

[chrislovecnm]

Also, we are creating ordering problem. The network will depend on a cluster, and I would like for this to not. IG and secrets depend on a cluster, and it is a pain.

If you create a secret before a cluster it is not handled gracefully.

[geojaz]

So we had an idea on the call about how to make this less painful for those with existing clusters- essentially to allow the network elements to be specified in both the cluster spec and the network spec and let the cluster spec overload the network spec... just tossing around ideas for now.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[chrislovecnm]

/lifecycle frozen
/remove-lifecycle stale