Tpot affects the nb of jobs at import

[Karim-53]

Hi,

when importing tpot on my linux machine the nb of effective joblib jobs is reduced to 1
the issue is not happening on the windows machine

Process to reproduce the issue

import sys
import platform
print({
        "python_version": platform.python_version(),
        "python_implementation": platform.python_implementation(),
        "platform": platform.platform(),
        "system": platform.system(),
        "architecture": platform.architecture(),
        "processor": platform.processor(),
    })
import joblib
print('joblib.__version__', joblib.__version__)
from joblib import effective_n_jobs
print('step1 logic_processors n_jobs = ', effective_n_jobs(n_jobs=-1))
import tpot
print('tpot.__version__', tpot.__version__)
print('step2 logic_processors n_jobs = ', effective_n_jobs(n_jobs=-1))
assert effective_n_jobs(n_jobs=-1) > 1

Expected result

output on windows machine (no problem):

{'python_version': '3.11.4', 'python_implementation': 'CPython', 'platform': 'Windows-10-10.0.19045-SP0', 'system': 'Windows', 'architecture': ('64bit', 'WindowsPE'), 'processor': 'Intel64 Family 6 Model 140 Stepping 1, GenuineIntel'}
joblib.__version__ 1.3.2
step1 logic_processors n_jobs =  8
tpot.__version__ 0.12.2
step2 logic_processors n_jobs =  8

Current result

output on linux machine:

{'python_version': '3.11.5', 'python_implementation': 'CPython', 'platform': 'Linux-4.12.14-197.108-default-x86_64-with-glibc2.26', 'system': 'Linux', 'architecture': ('64bit', 'ELF'), 'processor': 'x86_64'}
joblib.__version__ 1.3.2
step1 logic_processors n_jobs =  56
tpot.__version__ 0.12.2
step2 logic_processors n_jobs =  1
Traceback (most recent call last):
  File "/dss/dsshome1/0E/di35cex/shapley-data-interaction/bug.py", line 18, in <module>
    assert effective_n_jobs(n_jobs=-1) > 1
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

Possible fix

no idea

[perib]

I tested this and it is working correctly on linux on my machine. I am not sure why it is producing a different output on your machine... when you get n_jobs to a positive value, does it work correctly then?

If you look at CPU utilization, does it seem to work as expected even if that function call didn't return the expected value?

Maybe try Python 3.10?

The only other thing I can think of is maybe some packages are out of date?