Fatal exception - DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23.

[ScottB129]

Advised to "import this functionality directly from joblib, which can be installed with: pip install joblib." However, I have tried this and still hit with the same error code.

Full error:
(.env) scott@scott-VirtualBox:~/freqtrade$ source .env/bin/activate; python freqtrade/main.py
/home/scott/freqtrade/.env/lib/python3.6/site-packages/sklearn/externals/joblib/__init__.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
  warnings.warn(msg, category=DeprecationWarning)
Fatal exception!
Traceback (most recent call last):
  File "freqtrade/main.py", line 42, in main
    config = Configuration(args).get_config()
  File "/home/scott/freqtrade/freqtrade/configuration.py", line 299, in get_config
    self.config = self.load_config()
  File "/home/scott/freqtrade/freqtrade/configuration.py", line 47, in load_config
    config = self._load_config_file(self.args.config)
  File "/home/scott/freqtrade/freqtrade/configuration.py", line 81, in _load_config_file
    conf = json.load(file)
  File "/usr/lib/python3.6/json/__init__.py", line 299, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 2 column 24 (char 25) 

• Operating system: Ubuntu
• Python Version: 3.6.8
• CCXT version: 1.18.1054

[hroff-1902]

It's not you, who should "import this functionality directly from joblib", it's the skopt (scikit-optimize) library, used by freqtrade for hyperoptimization, which should use joblib directly instead of importing it from underlying sklearn...

They have a number of issues on this:
scikit-optimize/scikit-optimize#776
scikit-optimize/scikit-optimize#774
, etc...

Exception you obtain is not related to the DeprecationWarning imho.

[xmatthias]

The problem you're having is a syntax error in your configuration:

probably line 2 (or 3) ...

json.decoder.JSONDecodeError: Expecting value: line 2 column 24 (char 25) 

[Kaidokuni]

Hey, I have the same problem when I try to do Hyperopt, I have this
"c:\programdata\anaconda3\lib\site-packages\sklearn\externals\joblib_init_.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
warnings.warn(msg, category=DeprecationWarning)"
even after doing "pip install joblib" it's still there, how can I solve this or "re-serialize those models". I didn't find any instructions clear enough to know what to do even in the references @hroff-1902 gave.

[xmatthias]

it's a warning that will currently appear for everyone until sklearn fixes it. However, it's a warning, nothing that's breaking your process - so if you have a problem - this is not the cause of it.

[Kaidokuni]

Because it appears between almost every epoch result, I thought it was slowing down the process somehow.

[Kaidokuni]

Like this and then the hyperopt shut down
"..............c:\programdata\anaconda3\lib\site-packages\sklearn\externals\joblib_init_.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
warnings.warn(msg, category=DeprecationWarning)
................................c:\programdata\anaconda3\lib\site-packages\sklearn\externals\joblib_init_.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
warnings.warn(msg, category=DeprecationWarning)
....................c:\programdata\anaconda3\lib\site-packages\sklearn\externals\joblib_init_.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
warnings.warn(msg, category=DeprecationWarning)
....................c:\programdata\anaconda3\lib\site-packages\sklearn\externals\joblib_init_.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
warnings.warn(msg, category=DeprecationWarning)
..............c:\programdata\anaconda3\lib\site-packages\sklearn\externals\joblib_init_.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
warnings.warn(msg, category=DeprecationWarning)
2019-09-22 18:06:01,344 - freqtrade - ERROR - Fatal exception!
joblib.externals.loky.process_executor._RemoteTraceback:
"""
Traceback (most recent call last):
File "c:\programdata\anaconda3\lib\site-packages\joblib-0.13.2-py3.7.egg\joblib\externals\loky\backend\queues.py", line 150, in feed
obj = dumps(obj, reducers=reducers)
File "c:\programdata\anaconda3\lib\site-packages\joblib-0.13.2-py3.7.egg\joblib\externals\loky\backend\reduction.py", line 243, in dumps
dump(obj, buf, reducers=reducers, protocol=protocol)
File "c:\programdata\anaconda3\lib\site-packages\joblib-0.13.2-py3.7.egg\joblib\externals\loky\backend\reduction.py", line 236, in dump
_LokyPickler(file, reducers=reducers, protocol=protocol).dump(obj)
File "c:\programdata\anaconda3\lib\site-packages\joblib-0.13.2-py3.7.egg\joblib\externals\cloudpickle\cloudpickle.py", line 267, in dump
return Pickler.dump(self, obj)
File "c:\programdata\anaconda3\lib\pickle.py", line 439, in dump
self.framer.end_framing()
File "c:\programdata\anaconda3\lib\pickle.py", line 198, in end_framing
self.commit_frame(force=True)
File "c:\programdata\anaconda3\lib\pickle.py", line 212, in commit_frame
write(FRAME + pack("<Q", len(data)))
MemoryError
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "c:\users\john\freqtrade\freqtrade\main.py", line 39, in main
args'func'
File "c:\users\john\freqtrade\freqtrade\optimize_init_.py", line 74, in start_hyperopt
hyperopt.start()
File "c:\users\john\freqtrade\freqtrade\optimize\hyperopt.py", line 404, in start
f_val = self.run_optimizer_parallel(parallel, asked)
File "c:\users\john\freqtrade\freqtrade\optimize\hyperopt.py", line 341, in run_optimizer_parallel
wrap_non_picklable_objects(self.generate_optimizer))(v) for v in asked)
File "c:\programdata\anaconda3\lib\site-packages\joblib-0.13.2-py3.7.egg\joblib\parallel.py", line 934, in call
self.retrieve()
File "c:\programdata\anaconda3\lib\site-packages\joblib-0.13.2-py3.7.egg\joblib\parallel.py", line 833, in retrieve
self._output.extend(job.get(timeout=self.timeout))
File "c:\programdata\anaconda3\lib\site-packages\joblib-0.13.2-py3.7.egg\joblib_parallel_backends.py", line 521, in wrap_future_result
return future.result(timeout=timeout)
File "c:\programdata\anaconda3\lib\concurrent\futures_base.py", line 425, in result
return self.__get_result()
File "c:\programdata\anaconda3\lib\concurrent\futures_base.py", line 384, in __get_result
raise self._exception
_pickle.PicklingError: Could not pickle the task to send it to the workers."

[xmatthias]

MemoryError

check out #2271 for this problem (it has nothing to do with the outputs though).

[guneet999]

C:\ProgramData\Anaconda3\envs\rasa\lib\site-packages\sklearn\externals\joblib_init_.py:15: FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
warnings.warn(msg, category=FutureWarning)

[hroff-1902]

Dear Sir (Madam, Sirs): We have received and read your interesting letter. The facts you relate are well known to science and are of no interest to it. Nevertheless we thank you warmly for your alertness and wish you success in your work and personal life.

😉

[Shivabhatt]

What is the solutuion of this error?

The error is:
"warnings.warn(msg, category=FutureWarning)
FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+."

[xmatthias]

it is no error but a warning (it's also not present in any current verision anymore - so i suspect you're using an old version of freqtrade).

[Shivabhatt]

it is no error but a warning (it's also not present in any current verision anymore - so i suspect you're using an old version of freqtrade).

How can i resolve the warning...

[xmatthias]

It is a warning - just ignore it.

[Shivabhatt]

It is a warning - just ignore it.

clf.fit = hog_features(X,y)
TypeError: 'numpy.ndarray' object is not callable

and for this error...what i do

[hroff-1902]

@Shivabhatt show me versions of scikit-learn, scikit-optimize and freqtrade you are using

[xmatthias]

Please don't highjack old issues, but open new issues if you're having problems instead. Highjacking issues will notify everyone who contributed in this issue.

Thanks.