Optimization multithreading broken since 0.6.0

[timhok]

Expected behavior

pip install backtesting==0.5.0
time python Squeeze.py
...
real    2m15.172s
user    31m49.496s
sys     0m8.278s

Code sample

from backtesting import Backtest
...
bt = Backtest(data, SqueezeStrategy, cash=10_000, commission=.001, margin=1)

stats = bt.optimize(
		bb_length = range(10, 31, 10),
		bb_std = [1.0, 2.0, 3.0, 4.0],
		kc_length = range(10, 31, 10),
		kc_mult = [1.0, 1.5, 2.0, 2.5],
		rsi_period = range(10, 23, 2),
		maximize='Equity Final [$]')
bt.plot(filename="plot.html")

Actual behavior

pip install backtesting==0.6.0
time python Squeeze.py
...
real    35m53.835s
user    36m41.256s
sys     0m43.006s

Additional info, steps to reproduce, full crash traceback, screenshots

I noticed that when upgrading from version 0.5.0 to 0.6.0 (and 0.6.1) optimization is taking way too long and the cpu usage is very low: running the same exact code on version 0.5.0 takes 2 minutes and uses all 16 cores of my cpu, and on version 0.6.0+ it takes 35 minutes, also "user mode" metric reported by "time" binary is almost the same as "real" which indicates task being run in single thread

Software versions

• Backtesting version: 0.6.0
• OS: Ubuntu 22.04.4 LTS
• Python version: 3.10.12

[kernc]

I noticed that when upgrading from version 0.5.0 to 0.6.0 (and 0.6.1) optimization is taking way too long

Could this be 0.6.1 only? 7b69b1f, where we switch to joblib.Parallel(prefer='threads'), is only present in 0.6.1.

backtesting.py/backtesting/backtesting.py

Lines 1498 to 1499 in 14f111a

	with Parallel(prefer='threads', require='sharedmem', max_nbytes='50M',
	n_jobs=-2, return_as='generator') as parallel:

I intended it to be overridable with joblib.parallel_config(backend=...), but I see it might be buggy with its require='sharedmem'.

Seeing joblib/joblib#1094, it would probably be just as well to avoid a joblib dependency and use multiprocessing straight as we used to. 🤔

pip install backtesting==0.6.0

If this is, in fact, 0.6.0, then it's a whole another issue!

[kernc]

Please try v0.6.2 that's just been released! 🙌

[timhok]

Please try v0.6.2 that's just been released! 🙌

Just tested - works like a charm, optimization load spreads across all cpu cores. Thank you so much! ❤️