[ENH] support for custom joblib backends in parallelization

[fkiraly]

Adds support for custom joblib backends in the sktime unified parallelization interface. Fixes #5535.

It seems that using custom backends was previously possible via parameter passing, albeit not properly tested as a possibility.

This PR adds back the possibility via a "joblib" parallelization mechanism.

I would be thankful if you could suggest how to best test this, @SmirnGregHM - preferably without adding soft dependencies.

[fkiraly]

@SmirnGregHM, have you had opportunity to test whether this works for your use case? And/or whether you have suggestions on changing this? I'd rather not merge and find out it does not work for you...

For instance, do you think that backend="joblib" should also work?

[SmirnGregHM]

@fkiraly sorry for the delay

I have just checked your solution and it works for me. Interface looks a little cumbersome to me though. I'd say, any joblib backend should be available with the same interface. It will also be consistent with joblib.Parallel interface. But I also see the benefits of having more options than just joblib and benefits of keeping the old interface of passing 'loky' or 'multiprocessing' to backend.

What would you think about renaming 'joblib_custom' to 'joblib'? Then one can do any of:

backend='threading', ... 

backend='joblib', backend_params={'backend': 'threading'}, ...

backend='joblib', backend_params={'backend': 'spark'}, ...

It works already, just with 'joblib_custom'

[fkiraly]

I see - that seems like an improvement!
This would also allow us, if we decide so, to remove direct specification of joblib backends.

[fkiraly]

done