epic: Complete InCluster Networking

[davidfestal]

TL;DR

Extend the limited In Cluster Network support provided in PR kcp-dev/kcp#1708, in order to add:

• Support for Syncing from multiple User Workspaces from given Syncer
• NetworkPolicy-based protection of the accesses to the CoreDNS services installed for each user orkspace

Progress tracking #

EPIC detailed issues and overall progress can be tracked with the following project view

Work Items

• Fix InCluster networking when Syncer syncs from multiple user workspaces kcp#1987
• Protect the access to the CoreDNS pods installed in the physical cluster through NetworkPolicies kcp#1988
• DNS objects should be garbage collected when their workspace is "deleted" #80
• Implement workspace-based isolation of downstream namespaces through NetworkPolicies  #113

[david-martin]

@davidfestal @sttts I was having a look through the codebase to get an understanding of how #113 might be solved by creating a NetworkPolicy in each namespace. One that would "lock down by default between workspaces, but allow namespace comm for the same workspace"

Some related discusssion in slack here https://kubernetes.slack.com/archives/C021U8WSAFK/p1667469578560679

A few thoughts:

• Is there a precedence in the code where some resource(s) are created in each namespace? I can see a kcp-default-token secret and kcp-root-ca.crt configmap whos name gets transformed when syncing, which leads to another question
• Would the NetworkPolicy be created by kcp in the upstream namespace, and synced to the downstream namespace(s), with transforms to ensure appropriate restrictions? Or would the NetworkPolicy be a 'special case' like Namespaces where they aren't synced, but instead created and managed in the downstream namespace(s) only?

My instinct is to create a default NetworkPolicy in every namespace that gets transformed when synced downstream.

[davidfestal]

@davidfestal @sttts I was having a look through the codebase to get an understanding of how #113 might be solved by creating a NetworkPolicy in each namespace. One that would "lock down by default between workspaces, but allow namespace comm for the same workspace"

Some related discusssion in slack here https://kubernetes.slack.com/archives/C021U8WSAFK/p1667469578560679

A few thoughts:

• Is there a precedence in the code where some resource(s) are created in each namespace? I can see a kcp-default-token secret and kcp-root-ca.crt configmap whos name gets transformed when syncing, which leads to another question
• Would the NetworkPolicy be created by kcp in the upstream namespace, and synced to the downstream namespace(s), with transforms to ensure appropriate restrictions? Or would the NetworkPolicy be a 'special case' like Namespaces where they aren't synced, but instead created and managed in the downstream namespace(s) only?

My instinct is to create a default NetworkPolicy in every namespace that gets transformed when synced downstream.

Here is the current take on this (still need to formalize it in an issue / EPIC):

• short term:
  • The syncer would create namespaced NetworkPolicies to ensure a strong network isolation of namespaces originating from different KCP workspaces (no syncing required here, since the policies restrict all traffix between those namespaces
  • User-provided NetworkPolicies (provided in KCP and synced to physical cluster namespaces) are NOT supported (merging both is not realistic, after thinking about it and trying to do so)
• mid/long term:
  • Use Admin Network Policies to isolation of namespaces originating from different KCP workspaces
  • Allow namespaced NetworkPolicies to be defined at the KCP level, and synced + transformed by the Syncer to enforce communication rules between namespaces inside the KCP workspace boundaries

We did not contemplate enabling the use-case when communications would be allowed between namespaces of originating from different KCP workspaces

[mjudeikis]

/transfer-issue contrib-tmc