Missing Network Policy Guidelines

[davi5e]

We use Linkerd in a cluster that pretty much blocks every INGRESS/EGRESS not white listed with NetworkPolicies or GlobalNetworkPolicies (via Calico's CRD).

After successfully upgrading Linkerd from 2.9.4 to 2.10.1 we can't figure out what the viz plugin need and the fact it's installed in its own namespace makes all our previous configuration useless...

Can anyone help with some guidelines on how to proceed? What ports are used to where? If a cluster-wide configuration is needed, what would it look like?

[adleong]

@davi5e off the top of my head, the only viz component that I can think of that should need egress access is the dashboard (i.e. the web deployment). I believe that other than the dashboard, all of core viz functionality should work without egress access. Does this match what you're seeing? Are there specific failures you're encountering in your cluster?

[davi5e]

@adleong I'll try to install it again Wednesday and post back more information/answers.

As for what I'm seeing, our "problem" is that all traffic is blocked by default due to a GlobalNetworkPolicy. In doing so, we need to painstakingly open every port for every workload, which ensures our security standards.

Setting this up is a huge pain, though. For example, we need to open EGRESS to kubernetes.default:443 in case some pod needs to talk to the control plane. We need to configure every INGRESS port and corresponding EGRESS communication...

Not knowing where to start makes the job all the more tedious since we'd rely on Calico's log report that tell us which packages are being dropped.

Anyhow, as I mentioned, I'll schedule some time to do this in 2 days.

EDIT: I had to reschedule the set up, hopefully will do it next week.

[adleong]

Ah, I hadn't realized that talking to the Kubernetes control plane would count as egress traffic for these purposes. Almost all Linkerd control plane components talk to the Kubernetes control plane. Anyway, I'm looking forward to seeing your findings.

[adleong]

@davi5e just curious if you've had a chance to look into this. Is there anything actionable we can do to help on this, or should we close this issue for now?

[davi5e]

To determine any and all ports Linkerd Viz uses is a painstaking work and we are having trouble scheduling the time to do the deployment process.

Overall, Linkerd itself is working fine and the Viz component that is missing after the upgrade is still very much uninstalled... We will try to replicate the same network policies used in the linkerd namespace and cross our fingers, but generally I think we may be the only ones to have the Calico integration in production with block-all-traffic-by-default (maybe because GKE forces Istio to be used). It does amaze me that no one seems to request INGRESS/EGRESS port configurations, even though they became practical only using GlobalNetworkPolicies...

Anyhow, I'd say there is nothing actionable to be done. At first I thought someone would have a list of ports or hopefully a whole network policy configuration to paste here (or in the docs). As is, I can be the one to do this but I can not say when this issue will be tackled internally...

[adleong]

That's totally fair @davi5e. I'm closing this issue now, but please reach out if we can be of assistance!

[davi5e]

So, I finally had the time and refactored our Global network policy to what's shown below. Now viz works just fine!

apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: allow-linkerd-traffic
spec:
  egress:
  # Allow all egress FROM linkerd
  - action: Allow
    source:
      namespaceSelector: app.kubernetes.io/name in { 'linkerd', 'linkerd-cni', 'linkerd-viz' }
  # Allow egress FROM any pod TO linkerd
  - action: Allow
    destination:
      namespaceSelector: app.kubernetes.io/name in { 'linkerd', 'linkerd-cni', 'linkerd-viz' }
  ingress:
  # Allow ingress TO any pod FROM linkerd
  - action: Allow
    source:
      namespaceSelector: app.kubernetes.io/name in { 'linkerd', 'linkerd-cni', 'linkerd-viz' }
  # Allow ingress TO linkerd FROM any pod
  - action: Allow
    destination:
      namespaceSelector: app.kubernetes.io/name in { 'linkerd', 'linkerd-cni', 'linkerd-viz' }
  # Needs to be lower than regular NetworkPolicies so it gets a priority (1000)
  order: 500
  types:
  - Ingress
  - Egress

p.s.: apiVersion may differ since we are using the GKE integration.