Tasks do not respect thread limt

[constantinpape]

Tasks tend to spawn way more threads / processes than specified.
This might be related to the luigi scheduler or python subprocess.call?!

[constantinpape]

This does not seem to be a general issue, but rather some workflows which do not respect the thread limit set, or use more than one thread although they should be single threaded. E.g. the watershed workflow.

[constantinpape]

Turns out the most likely culprit is numpy:
numpy/numpy#11826
To fix it, either set the corresponding environment variables or use https://github.com/joblib/threadpoolctl.

[constantinpape]

This is indeed the issue.
For now, the solution is to use this function and call it before the numpy import, like here.