No daemonic with threads

[franchuterivera]

If moving to thread based dask workers, we do not need to concern ourselves with daemonic workers.

Below line can be removed. Nevertheless, the question remains if for our testing we should keep this, and for this I am talking about:

auto-sklearn/test/conftest.py

Line 167 in 13d1c06

dask.config.set({'distributed.worker.daemon': False})

Does it makes sense to still test with process==true for the user that wants to provide a process=true dask client?

[codecov]

Codecov Report

Merging #1018 (9673a48) into development (64193d2) will decrease coverage by 0.04%.
The diff coverage is n/a.

@@               Coverage Diff               @@
##           development    #1018      +/-   ##
===============================================
- Coverage        85.36%   85.32%   -0.05%     
===============================================
  Files              125      125              
  Lines             9848     9857       +9     
===============================================
+ Hits              8407     8410       +3     
- Misses            1441     1447       +6     

Impacted Files	Coverage Δ
autosklearn/automl.py	84.62% <ø> (-0.06%)	⬇️
...rn/pipeline/components/regression/random_forest.py	93.05% <0.00%> (-1.39%)	⬇️
...earn/pipeline/components/regression/extra_trees.py	92.50% <0.00%> (-1.25%)	⬇️
...ne/components/classification/passive_aggressive.py	94.04% <0.00%> (-1.20%)	⬇️
.../pipeline/components/classification/extra_trees.py	92.04% <0.00%> (-1.14%)	⬇️
autosklearn/pipeline/components/regression/sgd.py	95.78% <0.00%> (-1.06%)	⬇️
...osklearn/pipeline/components/classification/sgd.py	95.83% <0.00%> (-1.05%)	⬇️
...ipeline/components/regression/gradient_boosting.py	90.38% <0.00%> (-0.97%)	⬇️
...ine/components/classification/gradient_boosting.py	92.17% <0.00%> (-0.87%)	⬇️
...mponents/feature_preprocessing/no_preprocessing.py	95.00% <0.00%> (-0.24%)	⬇️
... and 7 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 64193d2...9673a48. Read the comment docs.

[mfeurer]

Great to see that this just works. I don't think we need to test for workers spawned with processes then (and you can also change the examples then)

[franchuterivera]

I think this is still needed for the examples, because when adding new workers, they are added as a new process. To scale on a local thread, one usually does the scale command of localcluster. Is it ok to leave it like this? or do you want to change the examples?

[mfeurer]

I think this is still needed for the examples, because when adding new workers, they are added as a new process.

Could you please try? If this is indeed the case, I think it would be good to add this as a comment to the example.

To scale on a local thread, one usually does the scale command of localcluster. Is it ok to leave it like this? or do you want to change the examples?

No, not with respect to that

[franchuterivera]

Could you please try? If this is indeed the case, I think it would be good to add this as a comment to the example.

Yes, this is needed in this context. Removing it causes multiple crashes. I have updated the example comment.