Loading resources for mouse is not working

[riederd]

Describe the bug
Loading MSigDB for mouse is not working:

$ python
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:08:06) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import logging
>>> import decoupler as dc
>>> logging.getLogger().setLevel(1)
>>> msigdb = dc.get_resource("MSigDB", organism="mouse")
Traceback (most recent call last):
  File "/home/rieder/.conda/envs/scirpy/lib/python3.10/site-packages/decoupler/omnip.py", line 61, in _check_if_omnipath
    import omnipath as op
  File "/home/rieder/.conda/envs/scirpy/lib/python3.10/site-packages/pypath/omnipath/__init__.py", line 23, in <module>
    from pypath.share import session as _session_mod
  File "/home/rieder/.conda/envs/scirpy/lib/python3.10/site-packages/pypath/__init__.py", line 110, in <module>
    info(0)
  File "/home/rieder/.conda/envs/scirpy/lib/python3.10/site-packages/pypath/__init__.py", line 81, in info
    _session_mod.log().msg(
AttributeError: 'NoneType' object has no attribute 'msg'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/rieder/.conda/envs/scirpy/lib/python3.10/site-packages/decoupler/omnip.py", line 254, in get_resource
    op = _check_if_omnipath()
  File "/home/rieder/.conda/envs/scirpy/lib/python3.10/site-packages/decoupler/omnip.py", line 65, in _check_if_omnipath
    raise ImportError(
ImportError: omnipath is not installed. Please install it by: `pip install omnipath`.

To Reproduce

1. Install the following packages:

$ pip install --upgrade git+https://github.com/saezlab/decoupler-py.git
$ pip install --upgrade git+https://github.com/saezlab/omnipath.git
$ pip install --upgrade git+https://github.com/saezlab/pypath.git

2. Run the following python code:

import logging
import decoupler as dc
logging.getLogger().setLevel(1)
msigdb = dc.get_resource("MSigDB", organism="mouse")

Expected behavior
I'd expect msigdb to be downloaded.

System

• OS: CentOS 7.9
• Python 3.10
• See above (git versions available on 2024-01-24 )

[PauBadiaM]

Hi @riederd,

Thanks for raising this issue. Unfortunately, OmniPath is currently under maintenance, @deeenes what is the status about it?

[Scheket]

Hi,
I have the same bug, due to: ImportError: cannot import name 'homology' from 'pypath.utils' and I see something in orthology:

['EnsemblOrtholog', 'EnsemblOrthology', 'HomologeneOrtholog', 'HomologeneOrthology', 'Literal', 'OmaOrtholog', 'OmaOrthology', 'OrthologBase', 'OrthologMeta', 'OrthologyManager', 'OrthologyTableKey', 'ProteinOrthology', 'Proteomes', 'PtmOrthology', 'SequenceContainer', 'builtins', 'cached', 'doc', 'file', 'loader', 'name', 'package', 'spec', '_const', '_log', '_logger', '_orthology_cleanup_timeloop', '_se', '_types', 'abc', 'annotations', 'biomart', 'cache_mod', 'collections', 'common', 'copy', 'curl', 'datetime', 'functools', 'get_df', 'get_dict', 'get_manager', 'homologene_input', 'imp', 'init', 'inspect', 'intera', 'iteritems', 'itertools', 'json', 'mapping', 'oma_input', 'os', 'pd', 'pickle', 'range', 're', 'session', 'settings', 'sys', 'taxonomy', 'time', 'timeloop', 'translate', 'translate_df', 'uniprot_db', 'uniprot_input', 'urls', 'xrange']

[deeenes]

Hello @riederd and @Scheket,

Apologies for showing up here so late! These issues are related to a major reorganization within pypath, the module that provides the orthology translation. I think with the release that I've just published today, it should work again. The same version should be also availble in PyPI (pip). And you also get better orthologous gene translation from now on, as pypath uses by default OMA (Orthologous Matrix) instead of the outdated NCBI HomoloGene. Let me know if you still experience issues.

Best,

Denes

[maximelepetit]

Hi @deeenes

Thanks a lot for this great tools !!

I want to use chicken or mouse version of Progeny, msigdb and collectri.

I upgrade the latest version of decoupler , omnipath and pypath using :

pip install --upgrade git+https://github.com/saezlab/decoupler-py.git
pip install --upgrade git+https://github.com/saezlab/omnipath.git
pip install --upgrade git+https://github.com/saezlab/pypath.git

I have the same error than @Scheket

For chicken i run :
progeny = dc.get_progeny(organism='9031', top=2000)
or
msigdb = dc.get_resource('MSigDB',organism='9031')
or
net = dc.get_collectri(organism='9031', split_complexes=False)

For mouse i run :
progeny = dc.get_progeny(organism='mouse', top=2000)
or
msigdb = dc.get_resource('MSigDB',organism='mouse')

Error :
ImportError : cannot import name 'homology' from 'pypath.utils' :

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
File ~/miniconda3/envs/decoupler/lib/python3.11/site-packages/decoupler/omnip.py:544, in translate_net(net, columns, source_organism, target_organism, id_type, unique_by, **kwargs)
    543 import pypath
--> 544 from pypath.utils import homology
    545 from pypath.share import common

ImportError: cannot import name 'homology' from 'pypath.utils' (/home/labex-cortex/miniconda3/envs/decoupler/lib/python3.11/site-packages/pypath/utils/__init__.py)

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
Cell In[5], line 1
----> 1 msigdb = dc.get_resource('MSigDB',organism=9031)
      2 msigdb

File ~/miniconda3/envs/decoupler/lib/python3.11/site-packages/decoupler/omnip.py:229, in get_resource(name, organism, **kwargs)
    225 df = df.drop(columns=['record_id', 'uniprot', 'entity_type', 'source'])
    227 if not _is_human(organism):
--> 229     df = translate_net(
    230         df,
    231         target_organism=organism,
    232         columns='genesymbol',
    233         unique_by=None,
    234     )
    236 return df.reset_index(drop=True)

File ~/miniconda3/envs/decoupler/lib/python3.11/site-packages/decoupler/omnip.py:560, in translate_net(net, columns, source_organism, target_organism, id_type, unique_by, **kwargs)
    553         raise RuntimeError(
    554             'The installed version of pypath-omnipath is too old, '
    555             f'the oldest compatible version is {PYPATH_MIN_VERSION}.'
    556         )
    558 except Exception:
--> 560     raise ImportError(
    561         'pypath-omnipath is not installed. Please install it with: '
    562         'pip install git+https://github.com/saezlab/pypath.git'
    563     )
    565 _source_organism = taxonomy.ensure_ncbi_tax_id(source_organism)
    566 _target_organism = taxonomy.ensure_ncbi_tax_id(target_organism)

ImportError: pypath-omnipath is not installed. Please install it with: pip install git+https://github.com/saezlab/pypath.git

For mouse Collectri work :
net = dc.get_collectri(organism='mouse', split_complexes=False)

Best.

Maxime

[maximelepetit]

Hi !!
The ImportError : cannot import name 'homology' from 'pypath.utils' was fix by updating the latest version:

omnipath : 1.0.8
pypath-omnipath : 0.16.9
decoupler : 1.6.0
python : 3.11.5
System : Ubuntu 20.04

I have a new error :

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
File ~/miniconda3/envs/decoupler/lib/python3.11/site-packages/decoupler/omnip.py:662, in translate_net(net, columns, source_organism, target_organism, id_type, unique_by, **kwargs)
    661         _misc.log_traceback(msg)
--> 662         raise RuntimeError(msg)
    664 except Exception:

RuntimeError: The installed version of pypath-omnipath is too old, the oldest compatible version is 0.14.28.

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
Cell In[4], line 1
----> 1 progeny = dc.get_progeny(organism='mouse', top=500)
      2 progeny

File ~/miniconda3/envs/decoupler/lib/python3.11/site-packages/decoupler/omnip.py:214, in get_progeny(organism, top, **kwargs)
    210 p = op._misc.dtypes.auto_dtype(p)
    212 if not _is_human(organism):
--> 214     p = translate_net(
    215         p,
    216         columns='target',
    217         source_organism=9606,
    218         target_organism=organism,
    219     )
    221 return p.reset_index(drop=True)

File ~/miniconda3/envs/decoupler/lib/python3.11/site-packages/decoupler/omnip.py:671, in translate_net(net, columns, source_organism, target_organism, id_type, unique_by, **kwargs)
    666     msg = (
    667         'pypath-omnipath is not installed. Please install it with: '
    668         'pip install git+https://github.com/saezlab/pypath.git'
    669     )
    670     _misc.log_traceback(msg)
--> 671     raise ImportError(msg)
    673 _source_organism = taxonomy.ensure_ncbi_tax_id(source_organism)
    674 _target_organism = taxonomy.ensure_ncbi_tax_id(target_organism)

ImportError: pypath-omnipath is not installed. Please install it with: pip install git+https://github.com/saezlab/pypath.git

Log :

DEBUG:root:Initialized `<Downloader[options=Options(url='https://omnipathdb.org', fallback_urls=('http://no-tls.omnipathdb.org',), static_url='http://no-tls.static.omnipathdb.org/resources', license=None, cache=<FileCache[size=7, path='/home/labex-cortex/.cache/omnipathdb']>, autoload=True, convert_dtypes=True, num_retries=3, timeout=600.0, chunk_size=8196)]>`
INFO:root:Downloading annotations for all proteins from the following resources: `['PROGENy']`
DEBUG:root:Unable to perform parameter validation for `resources`, haystack is empty
DEBUG:root:Attempting server `https://omnipathdb.org/`.
DEBUG:root:Looking up in cache: `https://omnipathdb.org/annotations?format=tsv&resources=PROGENy` ('0c5f7f75a74a2bdadf2e7efebe01ff1f').
DEBUG:root:Found data in cache `<FileCache[size=7, path='/home/labex-cortex/.cache/omnipathdb']>['0c5f7f75a74a2bdadf2e7efebe01ff1f']`
WARNING:root:NoneType: None

WARNING:root:The installed version of pypath-omnipath is too old, the oldest compatible version is 0.14.28.
/home/labex-cortex/miniconda3/envs/decoupler/lib/python3.11/site-packages/decoupler/_misc.py:22: UserWarning: The installed version of pypath-omnipath is too old, the oldest compatible version is 0.14.28.
  warnings.warn(msg)
WARNING:root:Traceback (most recent call last):
  File "/home/labex-cortex/miniconda3/envs/decoupler/lib/python3.11/site-packages/decoupler/omnip.py", line 662, in translate_net
    raise RuntimeError(msg)
RuntimeError: The installed version of pypath-omnipath is too old, the oldest compatible version is 0.14.28.

WARNING:root:pypath-omnipath is not installed. Please install it with: pip install git+https://github.com/saezlab/pypath.git
/home/labex-cortex/miniconda3/envs/decoupler/lib/python3.11/site-packages/decoupler/_misc.py:22: UserWarning: pypath-omnipath is not installed. Please install it with: pip install git+https://github.com/saezlab/pypath.git
  warnings.warn(msg)

I try to clean the cache and have the same error

Maxime

[deeenes]

Hi Maxime, What version of pypath-omnipath do you have installed?

import pypath
pypath.__version__
'0.16.4'

[maximelepetit]

Hi @deeenes ,

I list all the packages version here :

# packages in environment at /home/labex-cortex/miniconda3/envs/decoupler:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main    anaconda
_openmp_mutex             5.1                       1_gnu    anaconda
adjusttext                1.0.4                    pypi_0    pypi
anndata                   0.10.5.post1             pypi_0    pypi
annotated-types           0.6.0                    pypi_0    pypi
array-api-compat          1.4.1                    pypi_0    pypi
asttokens                 2.4.1                    pypi_0    pypi
attrs                     23.2.0                   pypi_0    pypi
backcall                  0.2.0              pyhd3eb1b0_0    anaconda
bcrypt                    4.1.2                    pypi_0    pypi
beautifulsoup4            4.12.3                   pypi_0    pypi
boltons                   23.1.1                   pypi_0    pypi
bzip2                     1.0.8                h7b6447c_0    anaconda
c-ares                    1.19.1               h5eee18b_0  
ca-certificates           2023.08.22           h06a4308_0    anaconda
certifi                   2024.2.2                 pypi_0    pypi
cffi                      1.16.0                   pypi_0    pypi
charset-normalizer        3.3.2                    pypi_0    pypi
comm                      0.2.1                    pypi_0    pypi
contourpy                 1.2.0                    pypi_0    pypi
cryptography              42.0.4                   pypi_0    pypi
curl                      8.5.0                hdbd6064_0  
cycler                    0.12.1                   pypi_0    pypi
debugpy                   1.8.1                    pypi_0    pypi
decorator                 5.1.1              pyhd3eb1b0_0    anaconda
decoupler                 1.6.0                    pypi_0    pypi
dill                      0.3.8                    pypi_0    pypi
docrep                    0.3.2                    pypi_0    pypi
et-xmlfile                1.1.0                    pypi_0    pypi
executing                 2.0.1                    pypi_0    pypi
face                      22.0.0                   pypi_0    pypi
fonttools                 4.49.0                   pypi_0    pypi
future                    0.18.3                   pypi_0    pypi
glom                      23.5.0                   pypi_0    pypi
h5py                      3.10.0                   pypi_0    pypi
idna                      3.6                      pypi_0    pypi
inflect                   7.0.0                    pypi_0    pypi
ipykernel                 6.29.2                   pypi_0    pypi
ipython                   8.21.0                   pypi_0    pypi
jedi                      0.19.1                   pypi_0    pypi
joblib                    1.3.2                    pypi_0    pypi
jupyter-core              5.7.1                    pypi_0    pypi
jupyter_client            8.6.0           py311h06a4308_0    anaconda
kiwisolver                1.4.5                    pypi_0    pypi
krb5                      1.20.1               h143b758_1  
ld_impl_linux-64          2.38                 h1181459_1    anaconda
legendkit                 0.3.3                    pypi_0    pypi
libcurl                   8.5.0                h251f7ec_0  
libedit                   3.1.20230828         h5eee18b_0  
libev                     4.33                 h7f8727e_1  
libffi                    3.4.4                h6a678d5_0    anaconda
libgcc-ng                 11.2.0               h1234567_1    anaconda
libgomp                   11.2.0               h1234567_1    anaconda
libnghttp2                1.57.0               h2d74bed_0  
libsodium                 1.0.18               h7b6447c_0    anaconda
libssh2                   1.10.0               hdbd6064_2  
libstdcxx-ng              11.2.0               h1234567_1    anaconda
libuuid                   1.41.5               h5eee18b_0    anaconda
llvmlite                  0.42.0                   pypi_0    pypi
lxml                      5.1.0                    pypi_0    pypi
marsilea                  0.3.2                    pypi_0    pypi
matplotlib                3.8.3                    pypi_0    pypi
matplotlib-inline         0.1.6           py311h06a4308_0    anaconda
natsort                   8.4.0                    pypi_0    pypi
ncurses                   6.4                  h6a678d5_0    anaconda
nest-asyncio              1.6.0                    pypi_0    pypi
networkx                  3.2.1                    pypi_0    pypi
numba                     0.59.0                   pypi_0    pypi
numpy                     1.26.4                   pypi_0    pypi
omnipath                  1.0.8                    pypi_0    pypi
openpyxl                  3.1.2                    pypi_0    pypi
openssl                   3.0.12               h7f8727e_0    anaconda
packaging                 23.2                     pypi_0    pypi
pandas                    2.2.0                    pypi_0    pypi
paramiko                  3.4.0                    pypi_0    pypi
parso                     0.8.3              pyhd3eb1b0_0    anaconda
patsy                     0.5.6                    pypi_0    pypi
pexpect                   4.9.0                    pypi_0    pypi
pickleshare               0.7.5           pyhd3eb1b0_1003    anaconda
pillow                    10.2.0                   pypi_0    pypi
pip                       24.0                     pypi_0    pypi
pip-review                1.3.0                    pypi_0    pypi
platformdirs              4.2.0                    pypi_0    pypi
prompt-toolkit            3.0.43                   pypi_0    pypi
psutil                    5.9.8                    pypi_0    pypi
ptyprocess                0.7.0              pyhd3eb1b0_2    anaconda
pure_eval                 0.2.2              pyhd3eb1b0_0    anaconda
pycparser                 2.21                     pypi_0    pypi
pycurl                    7.45.3                   pypi_0    pypi
pydantic                  2.6.1                    pypi_0    pypi
pydantic-core             2.16.2                   pypi_0    pypi
pydeseq2                  0.4.4                    pypi_0    pypi
pygments                  2.17.2                   pypi_0    pypi
pynacl                    1.5.0                    pypi_0    pypi
pynndescent               0.5.11                   pypi_0    pypi
pyparsing                 3.1.1                    pypi_0    pypi
pypath                    0.1                      pypi_0    pypi
pypath-common             0.2.0                    pypi_0    pypi
pypath-omnipath           0.16.9                   pypi_0    pypi
pyreadr                   0.5.0                    pypi_0    pypi
pysftp                    0.2.9                    pypi_0    pypi
python                    3.11.5               h955ad1f_0    anaconda
python-dateutil           2.8.2              pyhd3eb1b0_0    anaconda
pytz                      2024.1                   pypi_0    pypi
pyyaml                    6.0.1                    pypi_0    pypi
pyzmq                     25.1.2                   pypi_0    pypi
rdata                     0.10.0                   pypi_0    pypi
readline                  8.2                  h5eee18b_0    anaconda
requests                  2.31.0                   pypi_0    pypi
scanpy                    1.9.8                    pypi_0    pypi
scikit-learn              1.4.1.post1              pypi_0    pypi
scipy                     1.12.0                   pypi_0    pypi
seaborn                   0.13.2                   pypi_0    pypi
session-info              1.0.0                    pypi_0    pypi
six                       1.16.0             pyhd3eb1b0_1    anaconda
soupsieve                 2.5                      pypi_0    pypi
sqlite                    3.41.2               h5eee18b_0    anaconda
sqlparse                  0.4.4                    pypi_0    pypi
stack-data                0.6.3                    pypi_0    pypi
statsmodels               0.14.1                   pypi_0    pypi
stdlib-list               0.10.0                   pypi_0    pypi
tabulate                  0.9.0                    pypi_0    pypi
threadpoolctl             3.3.0                    pypi_0    pypi
timeloop                  1.0.2                    pypi_0    pypi
tk                        8.6.12               h1ccaba5_0    anaconda
toml                      0.10.2                   pypi_0    pypi
tornado                   6.4                      pypi_0    pypi
tqdm                      4.66.2                   pypi_0    pypi
traitlets                 5.14.1                   pypi_0    pypi
typing-extensions         4.9.0                    pypi_0    pypi
tzdata                    2024.1                   pypi_0    pypi
umap-learn                0.5.5                    pypi_0    pypi
urllib3                   2.2.1                    pypi_0    pypi
wcwidth                   0.2.13                   pypi_0    pypi
wrapt                     1.16.0                   pypi_0    pypi
xarray                    2024.2.0                 pypi_0    pypi
xlrd                      2.0.1                    pypi_0    pypi
xz                        5.4.2                h5eee18b_0    anaconda
zeromq                    4.3.4                h2531618_0    anaconda
zlib                      1.2.13               h5eee18b_0    anaconda

Pypath seems to be version 0.1 instead of pypath-omnipath have version 0.16.9.

I tried upgrade with :
pip3 install pypath --upgrade

They don't work :
Requirement already satisfied: pypath in ./miniconda3/envs/decoupler/lib/python3.11/site-packages (0.1)

Maxime

[maximelepetit]

hi @deeenes ,

I found the error !

In lib/python3.11/site-packages/, i had pypath_omnipath-0.16.9.dist-info/ and pypath-0.1.dist-info/ repertories, remove the last one seems to fix my issue.

Sorry for the inconvenience !

Maxime

[deeenes]

The package in PyPI is called pypath-omnipath, not pypath, because a package with the latter name has existed since long time ago. As far as I'm aware, this latter is not a really relevant package, I doubt anyone actually uses it. The name of our module is still pypath.