ImportError: cannot import name '_UnstableArchMixin'

[sreev]

IBM Watson Studio:Version 1.1.0-151 (1.1.0-151) on macOS Catalina 10.15.4

from sklearn.preprocessing import Normalizer
from sklearn.tree import DecitionTreeRegressor as Tree
from lale.lib.lale import Hyperopt


---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-28-2eee442a0b4d> in <module>
----> 1 from lale.lib.lale import Hyperopt

~/WatsonStudioDesktop/miniconda3/envs/desktop/lib/python3.6/site-packages/lale/lib/lale/__init__.py in <module>
     61 from .baseline_classifier import BaselineClassifier
     62 from .baseline_regressor import BaselineRegressor
---> 63 from .grid_search_cv import GridSearchCV
     64 from .hyperopt import Hyperopt
     65 from .topk_voting_classifier import TopKVotingClassifier

~/WatsonStudioDesktop/miniconda3/envs/desktop/lib/python3.6/site-packages/lale/lib/lale/grid_search_cv.py in <module>
     15 from typing import Any, Dict
     16 
---> 17 import lale.lib.sklearn
     18 import lale.search.lale_grid_search_cv
     19 import lale.operators

~/WatsonStudioDesktop/miniconda3/envs/desktop/lib/python3.6/site-packages/lale/lib/sklearn/__init__.py in <module>
    130 from .extra_trees_classifier import ExtraTreesClassifier
    131 from .extra_trees_regressor import ExtraTreesRegressor
--> 132 from .feature_agglomeration import FeatureAgglomeration
    133 from .function_transformer import FunctionTransformer
    134 from .gaussian_nb import GaussianNB

~/WatsonStudioDesktop/miniconda3/envs/desktop/lib/python3.6/site-packages/lale/lib/sklearn/feature_agglomeration.py in <module>
     13 # limitations under the License.
     14 
---> 15 import sklearn.cluster.hierarchical
     16 import lale.docstrings
     17 import lale.operators

~/WatsonStudioDesktop/miniconda3/envs/desktop/lib/python3.6/site-packages/sklearn/cluster/__init__.py in <module>
      4 """
      5 
----> 6 from .spectral import spectral_clustering, SpectralClustering
      7 from .mean_shift_ import (mean_shift, MeanShift,
      8                           estimate_bandwidth, get_bin_seeds)

~/WatsonStudioDesktop/miniconda3/envs/desktop/lib/python3.6/site-packages/sklearn/cluster/spectral.py in <module>
     15 from ..metrics.pairwise import pairwise_kernels
     16 from ..neighbors import kneighbors_graph
---> 17 from ..manifold import spectral_embedding
     18 from .k_means_ import k_means
     19 

~/WatsonStudioDesktop/miniconda3/envs/desktop/lib/python3.6/site-packages/sklearn/manifold/__init__.py in <module>
      3 """
      4 
----> 5 from .locally_linear import locally_linear_embedding, LocallyLinearEmbedding
      6 from .isomap import Isomap
      7 from .mds import MDS, smacof

~/WatsonStudioDesktop/miniconda3/envs/desktop/lib/python3.6/site-packages/sklearn/manifold/locally_linear.py in <module>
     10 from scipy.sparse.linalg import eigsh
     11 
---> 12 from ..base import BaseEstimator, TransformerMixin, _UnstableArchMixin
     13 from ..utils import check_random_state, check_array
     14 from ..utils.extmath import stable_cumsum

ImportError: cannot import name '_UnstableArchMixin'

[sreev]

temporary fix:
edited this file and removed the reference of '_UnstableArchMixin' in two places. one from import and another from method signature.

~/WatsonStudioDesktop/miniconda3/envs/desktop/lib/python3.6/site-packages/sklearn/manifold/locally_linear.py

[hirzel]

I'm glad you found a temporary work-around. From a first glance, this seems to arise from some interaction between Lale, Watson Studio Desktop, and sklearn. You already listed the WSD version, would you mind also reporting the sklearn and Lale versions? (In a notebook, you should be able to get that via !pip show scikit-learn and !pip show lale).

[sreev]

Name: scikit-learn
Version: 0.22.2.post1
Summary: A set of python modules for machine learning and data mining
Home-page: http://scikit-learn.org
Author: None
Author-email: None
License: new BSD
Location: /Users/svaddi/WatsonStudioDesktop/miniconda3/envs/desktop/lib/python3.6/site-packages
Requires: numpy, joblib, scipy
Required-by: sklearn, shap, lime, lightgbm, lale, aix360
Name: lale
Version: 0.3.11
Summary: Library for Semi-Automated Data Science
Home-page: https://github.com/IBM/lale
Author: Guillaume Baudart, Martin Hirzel, Kiran Kate, Parikshit Ram, Avraham Shinnar
Author-email: None
License: UNKNOWN
Location: /Users/svaddi/WatsonStudioDesktop/miniconda3/envs/desktop/lib/python3.6/site-packages
Requires: scipy, jsonsubschema, astunparse, pandas, decorator, xgboost, h5py, numpy, jsonschema, graphviz, lightgbm, hyperopt, scikit-learn
Required-by: 

[sreev]

WatsonStudioDesktop doesn't allow upgrade of a few packages ?
WatsonStudioDesktop has 'open case' option that is not working for me.
The reason to upgrade is if matching dependency versions falls into required/compatible versions.

!pip install pipdate
!pipdate

bitarray 0.8.3 -> 1.2.1...
ERROR: Cannot uninstall 'bitarray'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
bleach 3.1.0 -> 3.1.5...
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
bokeh 1.0.4 -> 2.0.2...
boto3 1.9.82 -> 1.13.0...
botocore 1.12.82 -> 1.16.0...
Bottleneck 1.2.1 -> 1.3.2...
certifi 2018.11.29 -> 2020.4.5.1...
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
ERROR: Cannot uninstall 'certifi'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
cffi 1.11.5 -> 1.14.0...
Click 7.0 -> 7.1.2...
cloudpickle 0.7.0 -> 1.4.1...
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
colorama 0.4.1 -> 0.4.3...
contextlib2 0.5.5 -> 0.6.0.post1...
cplex 12.8.0.1 -> 12.10.0.1...
cryptography 2.5 -> 2.9.2...
Cython 0.29.5 -> 0.29.17...
cytoolz 0.9.0.1 -> 0.10.1...
dask 1.1.1 -> 2.15.0...
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
decorator 4.3.2 -> 4.4.2...
ERROR: lale 0.3.11 has requirement scikit-learn==0.20.3, but you'll have scikit-learn 0.22.2.post1 which is incompatible.
defusedxml 0.5.0 -> 0.6.0...
distributed 1.25.3 -> 2.15.2...
docloud 1.0.315 -> 1.0.375...
docplex 2.8.125 -> 2.13.184...
docutils 0.14 -> 0.16...
ERROR: botocore 1.16.0 has requirement docutils<0.16,>=0.10, but you'll have docutils 0.16 which is incompatible.
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
ERROR: Cannot uninstall 'docutils'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
fastcache 1.0.2 -> 1.1.0...
filelock 3.0.10 -> 3.0.12...
Flask 1.0.2 -> 1.1.2...
Flask-Cors 3.0.7 -> 3.0.8...
future 0.17.1 -> 0.18.2...
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
gast 0.2.2 -> 0.3.3...
ERROR: tensorflow 2.1.0 has requirement gast==0.2.2, but you'll have gast 0.3.3 which is incompatible.
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
gevent 1.4.0 -> 20.5.0...
glob2 0.6 -> 0.7...
h5py 2.9.0 -> 2.10.0...
ERROR: lale 0.3.11 has requirement scikit-learn==0.20.3, but you'll have scikit-learn 0.22.2.post1 which is incompatible.
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
heapdict 1.0.0 -> 1.0.1...
ERROR: Cannot uninstall 'heapdict'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
hyperopt 0.2.3 -> 0.2.4...
ERROR: lale 0.3.11 has requirement hyperopt==0.2.3, but you'll have hyperopt 0.2.4 which is incompatible.
ERROR: lale 0.3.11 has requirement scikit-learn==0.20.3, but you'll have scikit-learn 0.22.2.post1 which is incompatible.
ibm-cos-sdk 2.4.2 -> 2.6.2...
ibm-cos-sdk-core 2.4.2 -> 2.6.2...
ibm-cos-sdk-s3transfer 2.4.2 -> 2.6.2...
idna 2.8 -> 2.9...
ERROR: requests 2.21.0 has requirement idna<2.9,>=2.5, but you'll have idna 2.9 which is incompatible.
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
imageio 2.4.1 -> 2.8.0...
ERROR: Cannot uninstall 'imageio'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
imagesize 1.1.0 -> 1.2.0...
ipykernel 4.8.2 -> 5.2.1...
ipython 7.2.0 -> 7.13.0...
ipywidgets 7.4.2 -> 7.5.1...
isort 4.3.4 -> 4.3.21...
jedi 0.13.2 -> 0.17.0...
Jinja2 2.10.3 -> 2.11.2...
jmespath 0.9.3 -> 0.9.5...
jsonschema 2.6.0 -> 3.2.0...
ERROR: lale 0.3.11 has requirement hyperopt==0.2.3, but you'll have hyperopt 0.2.4 which is incompatible.
ERROR: lale 0.3.11 has requirement scikit-learn==0.20.3, but you'll have scikit-learn 0.22.2.post1 which is incompatible.
jupyter-client 5.2.4 -> 6.1.3...
jupyter-console 6.0.0 -> 6.1.0...
jupyter-core 4.4.0 -> 4.6.3...
jupyterlab 0.35.3 -> 2.1.1...
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
jupyterlab-server 0.2.0 -> 1.1.1...
Keras 2.2.4 -> 2.3.1...
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
keyring 18.0.0 -> 21.2.1...
kiwisolver 1.0.1 -> 1.2.0...
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
libarchive-c 2.8 -> 2.9...
lief 0.9.0 -> 0.10.1...
llvmlite 0.27.0 -> 0.32.0...
ERROR: Cannot uninstall 'llvmlite'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
lxml 4.3.1 -> 4.5.0...
Markdown 3.0.1 -> 3.2.1...
MarkupSafe 1.1.0 -> 1.1.1...
matplotlib 3.0.2 -> 3.2.1...
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
more-itertools 5.0.0 -> 8.2.0...
msgpack 0.6.1 -> 1.0.0...
nbconvert 5.4.0 -> 5.6.1...
nbformat 4.4.0 -> 5.0.6...
networkx 2.2 -> 2.4...
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
nltk 3.4.5 -> 3.5...
notebook 5.6.0 -> 6.0.3...
numba 0.42.0 -> 0.49.0...
ERROR: Cannot uninstall 'llvmlite'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
numexpr 2.6.9 -> 2.7.1...
numpydoc 0.8.0 -> 0.9.2...
openpyxl 2.6.0 -> 3.0.3...
packaging 19.0 -> 20.3...
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
pandas 0.24.1 -> 1.0.3...
ERROR: lale 0.3.11 has requirement hyperopt==0.2.3, but you'll have hyperopt 0.2.4 which is incompatible.
ERROR: lale 0.3.11 has requirement pandas<=0.25.3, but you'll have pandas 1.0.3 which is incompatible.
ERROR: lale 0.3.11 has requirement scikit-learn==0.20.3, but you'll have scikit-learn 0.22.2.post1 which is incompatible.
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
parso 0.3.2 -> 0.7.0...
partd 0.3.9 -> 1.1.0...
path.py 11.5.0 -> 12.4.0...
pathlib2 2.3.3 -> 2.3.5...
pexpect 4.6.0 -> 4.8.0...
Pillow 5.4.1 -> 7.1.2...
ERROR: lime 0.2.0.0 has requirement pillow==5.4.1, but you'll have pillow 7.1.2 which is incompatible.
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
plotly 3.6.1 -> 4.6.0...
prometheus-client 0.5.0 -> 0.7.1...
prompt-toolkit 2.0.8 -> 3.0.5...
psutil 5.5.0 -> 5.7.0...
psycopg2 2.7.6.1 -> 2.8.5...
py 1.7.0 -> 1.8.1...
pyarrow 0.11.1 -> 0.17.0...
pycparser 2.19 -> 2.20...
pycurl 7.43.0.2 -> 7.43.0.5...
ERROR: Cannot uninstall 'pycurl'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
pyflakes 2.1.0 -> 2.2.0...
Pygments 2.3.1 -> 2.6.1...
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
pylint 2.2.2 -> 2.5.0...
pyodbc 4.0.25 -> 4.0.30...
pyOpenSSL 19.0.0 -> 19.1.0...
pyparsing 2.3.1 -> 2.4.7...
ERROR: lime 0.2.0.0 has requirement pillow==5.4.1, but you'll have pillow 7.1.2 which is incompatible.
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
PySocks 1.6.8 -> 1.7.1...
pytest-openfiles 0.4.0 -> 0.5.0...
python-dateutil 2.7.5 -> 2.8.1...
ERROR: lime 0.2.0.0 has requirement pillow==5.4.1, but you'll have pillow 7.1.2 which is incompatible.
ERROR: lale 0.3.11 has requirement hyperopt==0.2.3, but you'll have hyperopt 0.2.4 which is incompatible.
ERROR: lale 0.3.11 has requirement pandas<=0.25.3, but you'll have pandas 1.0.3 which is incompatible.
ERROR: lale 0.3.11 has requirement scikit-learn==0.20.3, but you'll have scikit-learn 0.22.2.post1 which is incompatible.
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
PyWavelets 1.0.1 -> 1.1.1...
ERROR: lime 0.2.0.0 has requirement pillow==5.4.1, but you'll have pillow 7.1.2 which is incompatible.
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
PyYAML 3.13 -> 5.3.1...
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
pyzmq 17.1.2 -> 19.0.0...
QtAwesome 0.5.6 -> 0.7.1...
qtconsole 4.4.3 -> 4.7.3...
QtPy 1.6.0 -> 1.9.0...
requests 2.21.0 -> 2.23.0...
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
rope 0.11.0 -> 0.16.0...
ruamel-yaml 0.15.46 -> 0.16.10...
ERROR: Cannot uninstall 'ruamel-yaml'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
s3transfer 0.1.13 -> 0.3.3...
scikit-image 0.14.1 -> 0.16.2...
ERROR: lime 0.2.0.0 has requirement pillow==5.4.1, but you'll have pillow 7.1.2 which is incompatible.
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
seaborn 0.9.0 -> 0.10.1...
snowballstemmer 1.2.1 -> 2.0.0...
ERROR: Cannot uninstall 'snowballstemmer'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
Sphinx 1.8.4 -> 3.0.3...
sphinxcontrib-websupport 1.1.0 -> 1.2.2...
SQLAlchemy 1.2.18 -> 1.3.16...
statsmodels 0.9.0 -> 0.11.1...
sympy 1.3 -> 1.5.1...
ERROR: Cannot uninstall 'sympy'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
tables 3.4.4 -> 3.6.1...
tabulate 0.8.2 -> 0.8.7...
tblib 1.3.2 -> 1.6.0...
tensorboard 2.1.1 -> 2.2.1...
ERROR: tensorflow 2.1.0 has requirement gast==0.2.2, but you'll have gast 0.3.3 which is incompatible.
ERROR: tensorflow 2.1.0 has requirement tensorboard<2.2.0,>=2.1.0, but you'll have tensorboard 2.2.1 which is incompatible.
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
tensorflow-estimator 2.1.0 -> 2.2.0...
ERROR: tensorflow 2.1.0 has requirement gast==0.2.2, but you'll have gast 0.3.3 which is incompatible.
ERROR: tensorflow 2.1.0 has requirement tensorboard<2.2.0,>=2.1.0, but you'll have tensorboard 2.2.1 which is incompatible.
ERROR: tensorflow 2.1.0 has requirement tensorflow-estimator<2.2.0,>=2.1.0rc0, but you'll have tensorflow-estimator 2.2.0 which is incompatible.
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
terminado 0.8.1 -> 0.8.3...
ERROR: Cannot uninstall 'terminado'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
testpath 0.4.2 -> 0.4.4...
toolz 0.9.0 -> 0.10.0...
tornado 4.5.3 -> 6.0.4...
tqdm 4.31.1 -> 4.45.0...
ERROR: lime 0.2.0.0 has requirement pillow==5.4.1, but you'll have pillow 7.1.2 which is incompatible.
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
traitlets 4.3.2 -> 4.3.3...
urllib3 1.24.1 -> 1.25.9...
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
wcwidth 0.1.7 -> 0.1.9...
Werkzeug 0.14.1 -> 1.0.1...
wheel 0.32.3 -> 0.34.2...
ERROR: tensorflow 2.1.0 has requirement gast==0.2.2, but you'll have gast 0.3.3 which is incompatible.
ERROR: tensorflow 2.1.0 has requirement tensorboard<2.2.0,>=2.1.0, but you'll have tensorboard 2.2.1 which is incompatible.
ERROR: tensorflow 2.1.0 has requirement tensorflow-estimator<2.2.0,>=2.1.0rc0, but you'll have tensorflow-estimator 2.2.0 which is incompatible.
ERROR: aix360 0.2.0 has requirement tensorflow==1.14, but you'll have tensorflow 2.1.0 which is incompatible.
widgetsnbextension 3.4.2 -> 3.5.1...
XlsxWriter 1.1.2 -> 1.2.8...
zict 0.1.3 -> 2.0.0...

[sreev]

WatsonStudioDesktop on Windows 10 failed integration with python, graphviz, etc...
I switched to Mac.

[hirzel]

Thanks again for reporting this. We usually run Lale either directly in Jupyter or VSCode, or in the cloud version of Watson Studio. But I have not tried WatsonStudioDesktop yet. So I am not sure how to diagnose the issue. Glad to see you have a temporary fix in place.