Update the proxy injector MWC to include reinvocationPolicy

[ihcsim]

K8s 1.15 introduces the reinvocationPolicy in the MWC and VWC allowing the webhook to be re-evaluated during a single admission request. This will ensure defaults defined by certain admission controllers like PodSecurityPolicy and LimitRanger are automatically added to the Linkerd proxy and proxy-init containers.

We should update the MWC template to include this property with the value IfNeeded, so that K8s 1.15+ users can take advantage of it.

[ihcsim]

To ensure backward compatibility, this property should only be added if users are on K8s 1.15+. The logic that copies the primary container's security context to the sidecar container in pkg/inject/inject.go should be retained, to ensure support of PSP continues to work.

[alpeb]

@ihcsim LimitRanger and PSP will participate during the reinvocation by default (after k8s 1.15) and will set the defaults on the sidecar, no config changes needed. We want to leave the injector's reinvocationPolicy to its default Never, as that property only affects the injector itself, not the other admission controllers.

[grampelberg]

@alpeb @ihcsim we should close this issue out right?

[ihcsim]

@alpeb I see what you mean - we will need to set this policy for our MWC if the proxy injector needs to be reinvoked. Closing this issue.

[jroper]

Not sure why this was closed. It would be good if linkerd did support IfNeeded - in Cloudstate, we inject a sidecar that needs to have some ports excluded from the service mesh using the config.linkerd.io/skip-outbound-ports annotation (this sidecar runs Akka, which forms a cluster between other pods of the same deployment, the cluster communication must bypass the service mesh because it by design cares about pod location to implement its sharding, replication and p2p communication patterns, but the service mesh thwarts this). So, if linkerd's webhook runs first, then the Cloudstate webhook runs, these annotations won't be picked up.

[alpeb]

@jroper that makes sense. Although currently when the proxy injector runs and detects the pod has already been injected, it'll bail out, so this will require extra modifications besides making reinvocationPolicy configurable. On the other hand, if I'm not mistaking, last time I checked I found custom webhooks were triggered alphabetically so maybe Cloudstate's would run first? (this isn't documented behavior though). Anyways supporting IfNeeded would be more robust; would your team be willing to tackle that change? I'd be glad to provide guidance.