Trouble setting up PyOD together with Keras and Tensorflow

[rwgsmits]

Hello,

I am very keen to use the PyOD models including the AutoEncoders. For this, I created a new conda environment with Python 3.7 and used the newest pip version to install pyod. This gives me a working environment. However, I am also interested in the AutoEncoders models, so used conda to install keras and tensorflow. After running the commands, as stated in the installation guide, it messes up the environment. Importing the pyod package then throws me the following error:

import pyod
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\pyod_init_.py", line 4, in
from . import utils
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\pyod\utils_init_.py", line 2, in
from .utility import check_parameter
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\pyod\utils\utility.py", line 14, in
import sklearn
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\sklearn_init_.py", line 80, in
from .base import clone
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\sklearn\base.py", line 21, in
from .utils import IS_32BIT
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\sklearn\utils_init.py", line 23, in
from .class_weight import compute_class_weight, compute_sample_weight
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\sklearn\utils\class_weight.py", line 7, in
from .validation import deprecate_positional_args
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\sklearn\utils\validation.py", line 25, in
from .fixes import object_dtype_isnan, parse_version
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\sklearn\utils\fixes.py", line 18, in
import scipy.stats
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\scipy\stats_init.py", line 388, in
from .stats import *
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\scipy\stats\stats.py", line 180, in
from . import distributions
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\scipy\stats\distributions.py", line 8, in
from .distn_infrastructure import (entropy, rv_discrete, rv_continuous,
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\scipy\stats_distn_infrastructure.py", line 23, in
from scipy import optimize
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\scipy\optimize_init.py", line 387, in
from .optimize import *
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\scipy\optimize\optimize.py", line 36, in
from .numdiff import approx_derivative
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\scipy\optimize_numdiff.py", line 6, in
from scipy.sparse.linalg import LinearOperator
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\scipy\sparse\linalg_init.py", line 114, in
from .eigen import *
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\scipy\sparse\linalg\eigen_init.py", line 9, in
from .arpack import *
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\scipy\sparse\linalg\eigen\arpack_init_.py", line 20, in
from .arpack import *
File "C:\Users\rsmits004\Anaconda3\envs\anomaly_test\lib\site-packages\scipy\sparse\linalg\eigen\arpack\arpack.py", line 43, in
from . import _arpack
ImportError: DLL load failed: The specified procedure could not be found.

I found that conda downgrades my scipy from 1.5.2. (pre-installation keras and tensorflow) to 1.5.0. (post-installation). I included the list of packages pre and post installation.

Packages before and after conda install keras tensorflow.docx

Uninstalling and installing the scipy package to 1.5.2. again seems to get importing pyod and tensorflow working again, but the follow message in the Anaconda Prompt makes me a bit nervous:

ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

tensorflow 2.3.0 requires numpy<1.19.0,>=1.16.0, but you'll have numpy 1.19.1 which is incompatible.
tensorflow 2.3.0 requires scipy==1.4.1, but you'll have scipy 1.5.2 which is incompatible.

Any recommendations on setting up the environment appropriately?

[andrewm4894]

iirc i got it working with Python 3.7 and:

Keras==2.3.1 
tensorflow==1.14.0

[yzhao062]

Yeah. We are leaving the setup of keras to the user.
I think the latest keras+TF works with my clound env.

My local env works with tf=1.14.0 and keras==2.2.4