Ensure NetworkPolicy for seed API server

[danielfoehrKn]

What this PR does / why we need it:
The gardenlet deploys Network Policies that should isolate the workload in the Shoot namespace in the seed from the Seed components.

This PR creates two controllers that are responsible for the creation and update of the network policy "allow-to-seed-apiserver".

Which issue(s) this PR fixes:
Fixes #2287

Special notes for your reviewer:
The dependency watchdog prober was relying on the allow-to-seed-apiserver NetworkPolicy which allowed a too broad set of ip ranges including public IPs. Now that the allow-to-seed-apiserver Policy only allows the Seed API server ip, the dependency watchdog prober needs the NetworkPolicy "allow-to-public".

@mvladev

Release note:

The NetworkPolicy 'allow-to-seed-apiserver'  has been improved to only allow the seed apiserver endpoint IPs. A new controller in the Gardenlet creates and updates the aforementioned NetworkPolicy. 

It is now possible to use a Kubernetes Cluster with private api server endpoints as a Seed cluster.

How to categorize this PR?

/area networking
/kind enhancement
/priority critical

[mvladev]

Few comments:

• There should be only 1 workqueue in which you'll put the namespaces in which the networkpolicy is effective - shoot and garden NS.
• The controller should listen for changes on Endpoints, Namespaces and NetworkPolicies.
• When kubernetes Endpoint is updated then it will add to the namespaceQueue all namespaces in which this networkpolicy is effective - shoot and garden NS. It's better to do n O(1) then O(n).
• When Namespace is added, it will add its name to the queue.
• When NetworkPolicy is added which is the one we'll reconcile, it will put the namespace in which it is in the naemspaceQueue.
• The controller would simply do a get for the kubernetes endpoint (using a lister / cached client), then try to check if the policy is correct / needs updates. it'll only operate on a single namespace (O(1)).

[danielfoehrKn]

@mvladev

There should be only 1 workqueue in which you'll put the namespaces in which the networkpolicy is effective - shoot and garden NS.

Sure makes sense. Atm. the endpoint worker updates all NetworkPolicies - by putting it on the namespace queue we could spread the load on multiple workers.

When NetworkPolicy is added which is the one we'll reconcile, it will put the namespace in which it is in the naemspaceQueue.

I would rather have, like it is implemented now, the controller creating the NetworkPolicy instead of the shoot reconciliation creating an empty NetworkPolicy. What would be the advantage of the latter?

The controller should listen for changes on Endpoints, Namespaces and NetworkPolicies.

What is the use case for also watching NetworkPolicies?. As described above I would like to create and update it in the controller instead of applying it via helm chart in the reconcile.

The only use case I see for that is when someone alters the NetworkPolicy so that we reconcile it back to the desired state. Is that a valid one or is the overhead to much because most likely that will not happen?

EDIT: Updated to only work on one the namespace queue.

[mvladev]

What is the use case for also watching NetworkPolicies?. As described above I would like to create and update it in the controller instead of applying it via helm chart in the reconcile.

The only use case I see for that is when someone alters the NetworkPolicy so that we reconcile it back to the desired state. Is that a valid one or is the overhead to much because most likely that will not happen?

Well the idea is that it'll always react on changes. If for example someone deletes this network policy, nothing will re-create it as nothing will add it to the queue. The only option would be to restart the controller or edit the namespace or kubernetes endpoint.

[danielfoehrKn]

@mvladev I was asking because for the core Kubernetes controllers do not watch Endpoints resources - so a manual edit of an endpoints resource will not be reconciled back to its desired state.
I can only suspect that a watch / informer for all endpoint resources in a large clusters is too costly (considering the benefits).
But I guess for the NetworkPolicies it might be ok.

I have now added a NetworkPolicy watch here.
I also added a pre-check in the namespace reconcile that prevents updates if the NetworkPolicy has already the desired state.
This prevents duplicate updates as the controller both creates / updates the NetworkPolicy but also watches it -> manual updates to the NetworkPolicy would otherwise trigger a second update.

[rfranzke]

Nice PR, looking forward to this feature! :)

[ghost]

Command "/ to keep the NetworkPolicy "allow-to-seed-apiserver" in sync." either unknown, wrongly parameterized or not permitted for issue/PR.

[rfranzke]

/area networking
/area security
/size l
/needs changes

[rfranzke]

/needs rebase

[danielfoehrKn]

PTAL @mvladev

[rfranzke]

/assign @mvladev @rfranzke

[mvladev]

Just a small nit

[danielfoehrKn]

I think the test needs another test run - an unrelated test failed due to timeout.
Can I do that via a command?

[rfranzke]

/lgtm