Dask not running in parallel

[AMCTeamMIS665ZA]

Just getting started with Dask. I was following along with a Dask array example from here: https://examples.dask.org/array.html and also a Matthew Rocklin video here: https://youtu.be/TQM9zIBzNBo.

I expected the example to use all 12 cores on my Windows server, but it only ever used one processor. I tried to increase the size of the example in case it was finishing quickly, but it just ran longer on only one processor.

Is there something else that needs to be done to have Dask run in parallel and use all the cores?

I have no code to submit other than the example code from the examples.dask.org and the YouTube video.

Not clear if this is a bug, but it did not work at all like the examples.

Environment:

Dask version: 2021.10.0

• Python version: 3.9.7
• Operating System: Windows 2019 Server
• Install method (conda, pip, source): Base install of Anaconda that included Dask

[bryanwweber]

Hi @AMCTeamMIS665ZA, welcome to Dask! This is more of a usage question than a bug report or feature request. We encourage people with such questions to ask them at the Dask Discourse 😄 Also, although the code is from examples.dask.org, can you please copy it into the post on Discourse so that the post is self-contained for any future readers, and in case the examples on the site change? Thank you!

[AMCTeamMIS665ZA]

I'm confused. I'm following an example that clearly shows parallel use with a simple example in Dask and it did not work as demonstrated. This could be a bug unless there are some steps in the examples that were not explained or documented. Without knowing if any steps were omitted from the example, I would conclude this is a bug.

[jrbourbeau]

That's a fair point @AMCTeamMIS665ZA, in that example we explicitly set n_workers and threads_per_worker in the first notebook cell:

client = Client(processes=False, threads_per_worker=4,
                n_workers=1, memory_limit='2GB')

IIUC this is so we don't crash binder resources when running examples interactively. If you remove those n_workers / threads_per_worker specifications, you should see more resources being used on your machine