Let Traefik's route traffic across namespaces via IngressRoute resources

[olivier-lacroix]

Hello,

Setting allowCrossNamespace=true fixes #568

see https://doc.traefik.io/traefik/providers/kubernetes-crd/#allowcrossnamespace

Not 100% sure of the security implication of that setting - would it be preferable to gate it behind a chart parameter?

Cheers,

Olivier

[consideRatio]

Ah nice investigative work! Is this tested to work already?

Btw, lots of test failures are unrelated, not sure if all are though.

[olivier-lacroix]

@consideRatio , yes, pulling the chart and patching it with that additional argument solves it for me.

[olivier-lacroix]

Indeed, the tests don't look too good!

[consideRatio]

@olivier-lacroix excellent okay!

Since users don't create IngressRoute objects themselves, but dask-gateway does it for them to specifically send traffic to the users created dask-cluster resources (scheduler/workers), I think we should be fine. If we didn't allow cross-namespaced work, I'd argue we should lower the permissions granted to traefik as well, but we have instead granted significant permissions.

dask-gateway/resources/helm/dask-gateway/templates/traefik/rbac.yaml

Lines 8 to 19 in 01eef76

	kind: ClusterRole
	apiVersion: rbac.authorization.k8s.io/v1
	metadata:
	name: {{ include "dask-gateway.traefikName" . }}
	rules:
	- apiGroups:
	- ""
	resources:
	- pods
	- services
	- endpoints
	- secrets

I think the intention was to support this by default historically, but with modern versions of traefik, allowing this by default changed in v2.3.5: traefik/traefik#7595

[olivier-lacroix]

@olivier-lacroix excellent okay!

Since users don't create IngressRoute objects themselves, but dask-gateway does it for them to specifically send traffic to the users created dask-cluster resources (scheduler/workers), I think we should be fine. If we didn't allow cross-namespaced work, I'd argue we should lower the permissions granted to traefik as well, but we have instead granted significant permissions.

dask-gateway/resources/helm/dask-gateway/templates/traefik/rbac.yaml

Lines 8 to 19 in 01eef76

	kind: ClusterRole
	apiVersion: rbac.authorization.k8s.io/v1
	metadata:
	name: {{ include "dask-gateway.traefikName" . }}
	rules:
	- apiGroups:
	- ""
	resources:
	- pods
	- services
	- endpoints
	- secrets

I think the intention was to support this by default historically, but with modern versions of traefik, allowing this by default changed in v2.3.5: traefik/traefik#7595

OK, great then!

[consideRatio]

Hmmm, one confusion point for me is case sensitivity.

Is this accepted? Looking at https://doc.traefik.io/traefik/providers/kubernetes-crd/#allowcrossnamespace, it could make more sense if this was --providers.kubernetesCRD.allowCrossNamespace=true or --providers.kubernetescrd.allowcrossnamespace=true in a way.

I wonder if there is a silent issue with this or other flags? I saw though in the logs of traefik that this was enabled, but with a warning asking if it was intentional or not, which seems weird if we explicitly try to set it.

I'm confused about how we pass these flags overall, hmmm...

[consideRatio]

I think everything is case insensitive, nevermind.