Messed up dca installation

[Rohit-Satyam]

Hi DCA developers

I am writing this as a humble request to fix the dca dependency issue related to tensorflow and keras. It seems like a never ending cycle of errors. Here is what I did and the error I am facing

mamba create -n enimpute -c bioconda dca tensorflow==2.4.0 keras==2.4.3

I am trying to create an environment using the abovementioned versions of tenserflow and keras and it gives me following error:

2023-07-07 20:05:16.990260: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/python/framework/dtypes.py:513: FutureWarning: In the future `np.object` will be defined as the corresponding NumPy scalar.
  np.object,
Traceback (most recent call last):
  File "/home/subudhak/miniconda3/envs/enimpute/bin/dca", line 10, in <module>
    sys.exit(main())
  File "/home/subudhak/miniconda3/envs/enimpute/lib/python3.8/site-packages/dca/__main__.py", line 145, in main
    import tensorflow as tf
  File "/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/python/__init__.py", line 46, in <module>
    from tensorflow.python import data
  File "/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/python/data/__init__.py", line 25, in <module>
    from tensorflow.python.data import experimental
  File "/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/python/data/experimental/__init__.py", line 96, in <module>
    from tensorflow.python.data.experimental import service
  File "/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/python/data/experimental/service/__init__.py", line 140, in <module>
    from tensorflow.python.data.experimental.ops.data_service_ops import distribute
  File "/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/python/data/experimental/ops/data_service_ops.py", line 25, in <module>
    from tensorflow.python.data.experimental.ops import compression_ops
  File "/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/python/data/experimental/ops/compression_ops.py", line 20, in <module>
    from tensorflow.python.data.util import structure
  File "/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/python/data/util/structure.py", line 26, in <module>
    from tensorflow.python.data.util import nest
  File "/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/python/data/util/nest.py", line 41, in <module>
    from tensorflow.python.framework import sparse_tensor as _sparse_tensor
  File "/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/python/framework/sparse_tensor.py", line 29, in <module>
    from tensorflow.python.framework import constant_op
  File "/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 29, in <module>
    from tensorflow.python.eager import execute
  File "/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 27, in <module>
    from tensorflow.python.framework import dtypes
  File "/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/python/framework/dtypes.py", line 513, in <module>
    np.object,
  File "/home/subudhak/.local/lib/python3.8/site-packages/numpy/__init__.py", line 305, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'object'.
`np.object` was a deprecated alias for the builtin `object`. To avoid this error in existing code, use `object` by itself. Doing this will not modify any behavior and is safe. 
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
Error in file(file, "rt") : cannot open the connection
In addition: Warning messages:
1: In dir.create("./DCA_result") : './DCA_result' already exists
2: In file(file, "rt") :
  cannot open file './DCA_result/result/mean.tsv': No such file or directory

Then I run pip install dca in enimpute environment but this upgrades the tenserflow to 2.4.4. The above error disappears and now a new error surfaces

2023-07-07 20:07:43.197533: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
Traceback (most recent call last):
  File "/home/subudhak/miniconda3/envs/enimpute/bin/dca", line 10, in <module>
    sys.exit(main())
  File "/home/subudhak/miniconda3/envs/enimpute/lib/python3.8/site-packages/dca/__main__.py", line 145, in main
    import tensorflow as tf
  File "/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/__init__.py", line 436, in <module>
    _ll.load_library(_main_dir)
  File "/home/subudhak/.local/lib/python3.8/site-packages/tensorflow/python/framework/load_library.py", line 153, in load_library
    py_tf.TF_LoadLibrary(lib)
tensorflow.python.framework.errors_impl.NotFoundError: /home/subudhak/miniconda3/envs/enimpute/lib/python3.8/site-packages/tensorflow/core/kernels/libtfkernel_sobol_op.so: undefined symbol: _ZNK10tensorflow8OpKernel11TraceStringB5cxx11ERKNS_15OpKernelContextEb
Error in file(file, "rt") : cannot open the connection

[Rohit-Satyam]

Having exhausted all the options using conda I finally was able to run it with the following commands

mamba create -n enimpute python==3.8
conda activate enimpute 
/home/subudhak/miniconda3/envs/enimpute/bin/pip3 install dca 'keras>=2.4,<2.6' 'tensorflow>=2.0,<2.5'

The key is to watch where the packages are being installed. In my case half of the dependencies were being picked up from .local/bin/dca so I first deleted the dca from bin.