Outdated "joblib" package causes error

[SouperDuper]

The "joblib" package updated to 1.2.0 on September 16th 2020 and is no longer compatible with the 1.0.1 version included in ENUNU.

Traceback (most recent call last):
  File "enunu.py", line 437, in <module>
  File "enunu.py", line 426, in main
  File "enunu.py", line 248, in main_as_plugin
  File "E:\UTAU\plugins\ENUNU-0.5.1\enulib\timing.py", line 201, in score2timing
    timelag = _score2timelag(config, score)
  File "E:\UTAU\plugins\ENUNU-0.5.1\enulib\timing.py", line 52, in _score2timelag
    in_scaler = joblib.load(config[typ].in_scaler_path)
  File "E:\UTAU\plugins\ENUNU-0.5.1\python-3.8.10-embed-amd64\lib\site-packages\joblib\numpy_pickle.py", line 585, in load
    obj = _unpickle(fobj, filename, mmap_mode)
  File "E:\UTAU\plugins\ENUNU-0.5.1\python-3.8.10-embed-amd64\lib\site-packages\joblib\numpy_pickle.py", line 504, in _unpickle
    obj = unpickler.load()
  File "pickle.py", line 1212, in load
KeyError: 10

To avoid incompatibility would it be possible to update the scikit-learn, numpy, and joblib packages in ENUNU to the current compatible versions? in the case of numpy, numpy<1.23
Users may not know they can update these packages themselves.

Thank you

[oatsu-gh]

Thank you reporting, let me check!

[oatsu-gh]

On my PC, v0.5.1 works properly without updatiing joblib.
Didn't you update any other libraries included in ENUNU?

Unfortunately, the latest scikit-learn is not compatible to ENUNU-0.5.1.

_____ξ ・ヮ・)ξ < ENUNU v0.5.1 ________
argv: ['enunu.py', 'C:\\Users\\CrazY\\AppData\\Local\\Temp\\utau1\\tmp1173.tmp']
2022-09-18 16:11:18.014900 : reading settings in TMP
2022-09-18 16:11:18.015899 : reading enuconfig
2022-09-18 16:11:18.027899 : copying UST
2022-09-18 16:11:18.028901 : copying Table
2022-09-18 16:11:18.029900 : converting UST to score with built-in function
2022-09-18 16:11:18.043901 : calculating timing with built-in function
Traceback (most recent call last):
  File "enunu.py", line 437, in <module>
  File "enunu.py", line 426, in main
  File "enunu.py", line 248, in main_as_plugin
  File "C:\Users\CrazY\AppData\Roaming\UTAU\plugins\ENUNU-0.5.1\enulib\timing.py", line 201, in score2timing
    timelag = _score2timelag(config, score)
  File "C:\Users\CrazY\AppData\Roaming\UTAU\plugins\ENUNU-0.5.1\enulib\timing.py", line 88, in _score2timelag
    lag = predict_timelag(
  File "C:\Users\CrazY\AppData\Roaming\UTAU\plugins\ENUNU-0.5.1\python-3.8.10-embed-amd64\lib\site-packages\nnsvs\gen.py", line 105, in predict_timelag
    timelag_linguistic_features = timelag_in_scaler.transform(
  File "C:\Users\CrazY\AppData\Roaming\UTAU\plugins\ENUNU-0.5.1\python-3.8.10-embed-amd64\lib\site-packages\sklearn\preprocessing\_data.py", line 509, in transform
    if self.clip:
AttributeError: 'MinMaxScaler' object has no attribute 'clip'
続行するには何かキーを押してください . . .

[SouperDuper]

Here is a particular model (polish language, trained by pixprucer) that does not work with joblib 1.0.1.
https://file.io/6rSFgMqdq9Yk
After updating the joblib package in enunu to 1.2.0 the model works. Alternatively one can train with an old joblib version.

interesting that the latest scikit-learn doesn't work as it work correctly, it works correctly on my side.

[SouperDuper]

I have recreated the issue personally with a japanese model and the results are the same.
Models trained with the latest joblib 1.2.0 package installed do not work with prior versions. However ENUNU with joblib 1.2.0 will work with models trained with older joblib.
I have trained a small subset of the No7 dataset on joblib versions 0.14.0, 0.17.0, 1.0.1, 1.1.0, and 1.2.0
https://file.io/QQGrtdzJ4wM0

To be sure I checked the compatibility of joblib verisons.

	MODEL	0.14.0	0.17.0	1.0.1	1.1.0	1.2.0
ENUNU
0.14.0		✓	✓	✓	✓	X
0.17.0		✓	✓	✓	✓	X
1.0.1		✓	✓	✓	✓	X
1.1.0		✓	✓	✓	✓	X
1.2.0		✓	✓	✓	✓	✓

Using joblib 1.2.0 with enunu doesn't break compatibility. While joblib 1.2.0 can use scalers from prior versions, scalers created with 1.2.0 do not appear to work with the older versions.

[oatsu-gh]

Thank you very much for the detailed research!!
With the joblib==1.2.0, latest numpy==1.22.4 and scikit-learn==0.23.2 ENUNU worked.

So I released ENUNU 0.5.2 containing these libraries!
https://github.com/oatsu-gh/ENUNU/releases/tag/v0.5.2

[SouperDuper]

It appears the zip file for enunu 0.5.2 still contains the outdated packages.

ENUNU-0.5.2>python-3.8.10-embed-amd64\python.exe -m pip show joblib numpy pyyaml
Name: joblib
Version: 1.0.1
Summary: Lightweight pipelining with Python functions
Home-page: https://joblib.readthedocs.io
Author: Gael Varoquaux
Author-email: gael.varoquaux@normalesup.org
License: BSD
Location: e:\utau\plugins\enunu-0.5.2\python-3.8.10-embed-amd64\lib\site-packages
Requires:
Required-by: librosa, scikit-learn
---
Name: numpy
Version: 1.21.1
Summary: NumPy is the fundamental package for array computing with Python.
Home-page: https://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email:
License: BSD
Location: e:\utau\plugins\enunu-0.5.2\python-3.8.10-embed-amd64\lib\site-packages
Requires:
Required-by: fastdtw, librosa, nnsvs, numba, pyworld, resampy, scikit-learn, scipy, tensorboard, torch, torchvision
---
Name: PyYAML
Version: 5.4.1
Summary: YAML parser and emitter for Python
Home-page: https://pyyaml.org/
Author: Kirill Simonov
Author-email: xi@resolvent.net
License: MIT
Location: e:\utau\plugins\enunu-0.5.2\python-3.8.10-embed-amd64\lib\site-packages
Requires:
Required-by: omegaconf

[oatsu-gh]

Sorry for the mistake. I re-uploaded that.

[SouperDuper]

Thank you, the updated version is working as expected!
Your work is very much appreciated