Issue running Prefect on dask cluster installed with helm

[marvin-robot]

alistair.hey: Im trying to use Prefect with a dask kubernetes cluster (installed with helm stable/dask) - however whenever i install prefect on the dask worker pods it upgrades distributed and then the cluster falls apart - any one got prefect working in this way?

chris: Hey Alistair - that’s interesting; what version of distributed are you working with? If the version falls below 1.26.1, then I could see why your package manager would upgrade it — if not though, we’ll have to think about possible workarounds here :face_with_monocle:

alistair.hey: i created this issue on dask-kubernetes before i worked out that it was my inclusion of prefect in the "EXTRA_PIP_PACKAGES" section of the config.yaml passed to helm. dask/dask-kubernetes#161

alistair.hey: stable/dask installs version 1.25.2

chris: Ah ok very interesting. So you need 1.25.2; I would need to revisit, but I believe we upgraded our minimum distributed requirement to 1.26.1 to include some distributed features that we needed in deployment.

chris: you might be able to install prefect with no-dependencies, and then install the other dependencies that prefect requires manually to temporarily work around this issue

alistair.hey: I dont need 1.25.2, thats just whats bundled on the devdask/dask:1.1.0 container that the chart uses

alistair.hey: think iv found a workaround actually

chris: ok great - what is it?

alistair.hey: if i install prefect on the scheduler through the config.yaml used by helm it upgrades distributed on the other pods to 1.28.1

chris: ahhh yes, very nice

alistair.hey: im just testing now

alistair.hey: yes - that appears to have worked. Thanks

chris: great!

chris: <@ULVA73B9P> archive “Issue running Prefect on dask cluster installed with helm”