AttributeError: module 'sklearn.externals'

[hcp4715]

Dear experts,

I really happy that there is a python package for psychophysiological data!

I am trying to run the tutorial codes (https://neurokit.readthedocs.io/en/latest/tutorials/Bio.html) on my PC, but encountered this error.

Seems that I used an newer sklearn package, which no longer has 'joblib' module, but not very sure. Your help will be highly appreciated.

Here is information about my settings:
OS: Win 10, running on AMD64

Packages:
Python 3.7.4
IPython 7.8.0
Neurokit 0.2.8
pandas 0.25.1
Numpy 1.16.4
seaborn 0.9.0
sklearn 0.21.2

More specifically, I run this code:
bio = nk.bio_process(ecg=df["ECG"], rsp=df["RSP"], eda=df["EDA"], add=df["Photosensor"], sampling_rate=100)

and here comes the error message:

AttributeError Traceback (most recent call last)
in
1 # process the signals
----> 2 bio = nk.bio_process(ecg=df["ECG"], rsp=df["RSP"], eda=df["EDA"], add=df["Photosensor"], sampling_rate=100)
3
4 # plot the processed dataframe, normalizing all variables
5 nk.z_score(bio["df"]).plot()

~\Anaconda3\envs\biosigPy\lib\site-packages\neurokit\bio\bio_meta.py in bio_process(ecg, rsp, eda, emg, add, sampling_rate, age, sex, position, ecg_filter_type, ecg_filter_band, ecg_filter_frequency, ecg_segmenter, ecg_quality_model, ecg_hrv_features, eda_alpha, eda_gamma, scr_method, scr_treshold, emg_names, emg_envelope_freqs, emg_envelope_lfreq, emg_activation_treshold, emg_activation_n_above, emg_activation_n_below)
123 # ECG & RSP
124 if ecg is not None:
--> 125 ecg = ecg_process(ecg=ecg, rsp=rsp, sampling_rate=sampling_rate, filter_type=ecg_filter_type, filter_band=ecg_filter_band, filter_frequency=ecg_filter_frequency, segmenter=ecg_segmenter, quality_model=ecg_quality_model, hrv_features=ecg_hrv_features, age=age, sex=sex, position=position)
126 processed_bio["ECG"] = ecg["ECG"]
127 if rsp is not None:

~\Anaconda3\envs\biosigPy\lib\site-packages\neurokit\bio\bio_ecg.py in ecg_process(ecg, rsp, sampling_rate, filter_type, filter_band, filter_frequency, segmenter, quality_model, hrv_features, age, sex, position)
117 # ===============
118 if quality_model is not None:
--> 119 quality = ecg_signal_quality(cardiac_cycles=processed_ecg["ECG"]["Cardiac_Cycles"], sampling_rate=sampling_rate, rpeaks=processed_ecg["ECG"]["R_Peaks"], quality_model=quality_model)
120 processed_ecg["ECG"].update(quality)
121 processed_ecg["df"] = pd.concat([processed_ecg["df"], quality["ECG_Signal_Quality"]], axis=1)

~\Anaconda3\envs\biosigPy\lib\site-packages\neurokit\bio\bio_ecg.py in ecg_signal_quality(cardiac_cycles, sampling_rate, rpeaks, quality_model)
355
356 if quality_model == "default":
--> 357 model = sklearn.externals.joblib.load(Path.materials() + 'heartbeat_classification.model')
358 else:
359 model = sklearn.externals.joblib.load(quality_model)

AttributeError: module 'sklearn.externals' has no attribute 'joblib'

[sangfrois]

Hey @hcp4715 !

have you imported sklearn in your env ?
it should look like this :

from sklearn.externals import joblib
 bio = nk.bio_process(ecg=df["ECG"], rsp=df["RSP"], eda=df["EDA"], add=df["Photosensor"], sampling_rate=100)

# plot the processed dataframe, normalizing all variables
 nk.z_score(bio["df"]).plot()

of course, you have to import Neurokit as nk and pandas as pd, as well, to use the functions and play around with the data. I've ran into this error and multiple times, untill i realized I just hadn't import the sklearn dependency.

Tell me if it works out for you

[louis-youpling]

For an easy way to fix this is to reverse to an earlier version of scikit-learn
pip install scikit-learn==0.20.0

A more definite would be to remove references to externals.joblib here

NeuroKit.py/neurokit/bio/bio_ecg.py

Line 357 in c958934

model = sklearn.externals.joblib.load(Path.materials() + 'heartbeat_classification.model')

and use joblib directly

if quality_model == "default":
    quality_model = Path.materials() + 'heartbeat_classification.model'
model = joblib.load(quality_model)

[DominiqueMakowski]

I am super sorry about all this, as I currently don't have time to fix the module as much as I'd like it! Thanks a lot for all the people finding the workarounds 😍 💪

In fact, as I already mentioned somewhere, I plan on starting a massive overhaul, i.e., reimplementing some of the best stuff from neurokit almost from scratch, soon. As the package's scope has become quite fuzzy over the years, the goal of the refactoring is to make it safer, more flexible, more powerful, more documented, and overall better. This will probably be happening in a separated repo first (NeuroKit2?), with the aim of one day replacing the current version.

If you're by any chances interested in being somehow involved in this, please do let me know (e.g. via email dom.makowski@gmail.com). I could set up the repo in which we could start discussing ideas, design, roadmap, bugs, priorities and so on ☺️

[hcp4715]

For an easy way to fix this is to reverse to an earlier version of scikit-learn
pip install scikit-learn==0.20.0

A more definite would be to remove references to externals.joblib here

NeuroKit.py/neurokit/bio/bio_ecg.py

Line 357 in c958934

model = sklearn.externals.joblib.load(Path.materials() + 'heartbeat_classification.model')

and use joblib directly

if quality_model == "default":
    quality_model = Path.materials() + 'heartbeat_classification.model'
model = joblib.load(quality_model)

Thanks, Louis. I think maybe install an early version will solve this.

[hcp4715]

Hey @hcp4715 !

have you imported sklearn in your env ?
it should look like this :

from sklearn.externals import joblib
 bio = nk.bio_process(ecg=df["ECG"], rsp=df["RSP"], eda=df["EDA"], add=df["Photosensor"], sampling_rate=100)

# plot the processed dataframe, normalizing all variables
 nk.z_score(bio["df"]).plot()

of course, you have to import Neurokit as nk and pandas as pd, as well, to use the functions and play around with the data. I've ran into this error and multiple times, untill i realized I just hadn't import the sklearn dependency.

Tell me if it works out for you

Dear François,

Thanks for your response, I tried, but I guess the problem is the version of sklearn ;-).

Because my data is mainly skin conductance, so I directly used eda process function, without testing bio_process further. BTW, the EDA data are processed very well.

[louis-youpling]

Some changes suggested there: 06890a4

[anniezhi]

For an easy way to fix this is to reverse to an earlier version of scikit-learn
pip install scikit-learn==0.20.0

A more definite would be to remove references to externals.joblib here

NeuroKit.py/neurokit/bio/bio_ecg.py

Line 357 in c958934

model = sklearn.externals.joblib.load(Path.materials() + 'heartbeat_classification.model')

and use joblib directly

if quality_model == "default":
    quality_model = Path.materials() + 'heartbeat_classification.model'
model = joblib.load(quality_model)

I have tried this, and the following error appears:

ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-6-a95faca0bef5> in <module>
      1 # ECG analysis - neurokit
----> 2 result_org_2 = nk.ecg_process(X_train[100,:][~np.isnan(X_train[100,:])],
      3                               sampling_rate=300)

~/opt/anaconda3/envs/ML/lib/python3.8/site-packages/neurokit/bio/bio_ecg.py in ecg_process(ecg, rsp, sampling_rate, filter_type, filter_band, filter_frequency, segmenter, quality_model, hrv_features, age, sex, position)
    118     # ===============
    119     if quality_model is not None:
--> 120         quality = ecg_signal_quality(processed_ecg["ECG"]["Cardiac_Cycles"], sampling_rate, quality_model=quality_model)
    121         processed_ecg["ECG"].update(quality)
    122 

~/opt/anaconda3/envs/ML/lib/python3.8/site-packages/neurokit/bio/bio_ecg.py in ecg_signal_quality(cardiac_cycles, sampling_rate, quality_model)
    353     if quality_model == "default":
    354         quality_model = Path.materials() + 'heartbeat_classification.model'
--> 355     model = joblib.load(quality_model)
    356 
    357     # Initialize empty dict

~/opt/anaconda3/envs/ML/lib/python3.8/site-packages/joblib/numpy_pickle.py in load(filename, mmap_mode)
    583                     return load_compatibility(fobj)
    584 
--> 585                 obj = _unpickle(fobj, filename, mmap_mode)
    586     return obj

~/opt/anaconda3/envs/ML/lib/python3.8/site-packages/joblib/numpy_pickle.py in _unpickle(fobj, filename, mmap_mode)
    502     obj = None
    503     try:
--> 504         obj = unpickler.load()
    505         if unpickler.compat_mode:
    506             warnings.warn("The file '%s' has been generated with a "

~/opt/anaconda3/envs/ML/lib/python3.8/pickle.py in load(self)
   1208                     raise EOFError
   1209                 assert isinstance(key, bytes_types)
-> 1210                 dispatch[key[0]](self)
   1211         except _Stop as stopinst:
   1212             return stopinst.value

~/opt/anaconda3/envs/ML/lib/python3.8/pickle.py in load_global(self)
   1524         module = self.readline()[:-1].decode("utf-8")
   1525         name = self.readline()[:-1].decode("utf-8")
-> 1526         klass = self.find_class(module, name)
   1527         self.append(klass)
   1528     dispatch[GLOBAL[0]] = load_global

~/opt/anaconda3/envs/ML/lib/python3.8/pickle.py in find_class(self, module, name)
   1575             elif module in _compat_pickle.IMPORT_MAPPING:
   1576                 module = _compat_pickle.IMPORT_MAPPING[module]
-> 1577         __import__(module, level=0)
   1578         if self.proto >= 4:
   1579             return _getattribute(sys.modules[module], name)[0]

ModuleNotFoundError: No module named 'sklearn.externals.joblib'

[DominiqueMakowski]

@anniezhi please consider using NeuroKit2 (https://github.com/neuropsychology/NeuroKit) which should work ☺️