distributed and DNS requests

[abast]

When will the scheduler, the workers or the client perform DNS requests? How many DNS requests are to be expected?

Does it depend on the way, the cluster is initialized?
E.g. is dask-worker --nthreads 1 --nprocs 80 my_server_name:8786 triggering much more requests than dask-worker --nthreads 1 --nprocs 80 my_server_ip:8786?

Background: I got a call from my IT department because one of the compute servers was doing ~1 request each millisecond. This stopped as soon as I shut down the jupyter notebook and the distributed cluster, which is why I have the questions mentioned above.

[mrocklin]

Thanks for reporting @abast

cc @pitrou

[pitrou]

Right now, I'd recommend that you always use IP addresses. This is something we should fix in the future.

[mrocklin]

Is there anything wrong with dask-worker doing a single DNS lookup and then using that when it instantiates the Worker?

[pitrou]

Nothing wrong, but we should probably integrate that logic at a lower level (so that it works all kinds of connections).