module not found when installed with conda_install()
#838
Comments
|
Here's my guess as to what's happening based on the shared code (with my comments above) I think a big part of the problem here is that either |
I don't think loompy depends on a specific version. Look what happens when I don't force the 'r-reticulate' environment: This is the weirdest part. First python is updated. Then it is downgraded. |
|
Can you try your example once more with the development version of reticulate? You can install it with e.g. I just made a couple changes to |
I just tried: Why does it say that the module is there already? Edit: I will try to force r-reticulate now. Edit: Nothing, even forcing the environment and removing the package did not make it work. |
|
Dang, thanks. I'll try to take a closer look today. |
|
Works for me: Created on 2020-09-03 by the reprex package (v0.3.0) |
|
Ok then it must be some issue on my side... I am horrendously frustrated right now. Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> library(reticulate)
> # create new environment
> conda_create("loompy-test")
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done
Solving environment: ...working... done
## Package Plan ##
environment location: /local-scratch/users/myuser/miniconda3/envs/loompy-test
added / updated specs:
- python
The following NEW packages will be INSTALLED:
_libgcc_mutex conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge
_openmp_mutex conda-forge/linux-64::_openmp_mutex-4.5-1_gnu
ca-certificates conda-forge/linux-64::ca-certificates-2020.6.20-hecda079_0
certifi conda-forge/linux-64::certifi-2020.6.20-py38h32f6830_0
ld_impl_linux-64 conda-forge/linux-64::ld_impl_linux-64-2.34-hc38a660_9
libffi conda-forge/linux-64::libffi-3.2.1-he1b5a44_1007
libgcc-ng conda-forge/linux-64::libgcc-ng-9.3.0-h24d8f2e_16
libgomp conda-forge/linux-64::libgomp-9.3.0-h24d8f2e_16
libstdcxx-ng conda-forge/linux-64::libstdcxx-ng-9.3.0-hdf63c60_16
ncurses conda-forge/linux-64::ncurses-6.2-he1b5a44_1
openssl conda-forge/linux-64::openssl-1.1.1g-h516909a_1
pip conda-forge/noarch::pip-20.2.2-py_0
python conda-forge/linux-64::python-3.8.5-h1103e12_7_cpython
python_abi conda-forge/linux-64::python_abi-3.8-1_cp38
readline conda-forge/linux-64::readline-8.0-he28a2e2_2
setuptools conda-forge/linux-64::setuptools-49.6.0-py38h32f6830_0
sqlite conda-forge/linux-64::sqlite-3.33.0-h4cf870e_0
tk conda-forge/linux-64::tk-8.6.10-hed695b0_0
wheel conda-forge/noarch::wheel-0.35.1-pyh9f0ad1d_0
xz conda-forge/linux-64::xz-5.2.5-h516909a_1
zlib conda-forge/linux-64::zlib-1.2.11-h516909a_1009
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
#
# To activate this environment, use
#
# $ conda activate loompy-test
#
# To deactivate an active environment, use
#
# $ conda deactivate
[1] "/local-scratch/users/myuser/miniconda3/envs/loompy-test/bin/python"
> # install loompy into that environment
> conda_install(envname = "loompy-test",
+ packages = "loompy",
+ channel = "bioconda")
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
Solving environment: ...working... done
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
Solving environment: ...working... done
## Package Plan ##
environment location: /local-scratch/users/myuser/miniconda3/envs/loompy-test
added / updated specs:
- loompy
- python
The following NEW packages will be INSTALLED:
h5py conda-forge/linux-64::h5py-2.10.0-nompi_py36hecadee3_104
hdf5 conda-forge/linux-64::hdf5-1.10.6-nompi_h3c11f04_101
libblas conda-forge/linux-64::libblas-3.8.0-17_openblas
libcblas conda-forge/linux-64::libcblas-3.8.0-17_openblas
libgfortran-ng conda-forge/linux-64::libgfortran-ng-7.5.0-hdf63c60_16
liblapack conda-forge/linux-64::liblapack-3.8.0-17_openblas
libopenblas conda-forge/linux-64::libopenblas-0.3.10-pthreads_hb3c22a3_4
loompy bioconda/noarch::loompy-2.0.16-py_0
numpy conda-forge/linux-64::numpy-1.19.1-py36h3849536_2
pandas conda-forge/linux-64::pandas-1.1.1-py36h831f99a_0
python-dateutil conda-forge/noarch::python-dateutil-2.8.1-py_0
pytz conda-forge/noarch::pytz-2020.1-pyh9f0ad1d_0
scipy conda-forge/linux-64::scipy-1.5.2-py36h3a855aa_0
six conda-forge/noarch::six-1.15.0-pyh9f0ad1d_0
typing bioconda/linux-64::typing-3.5.2.2-py36_0
The following packages will be DOWNGRADED:
certifi 2020.6.20-py38h32f6830_0 --> 2020.6.20-py36h9f0ad1d_0
python 3.8.5-h1103e12_7_cpython --> 3.6.11-h4d41432_2_cpython
python_abi 3.8-1_cp38 --> 3.6-1_cp36m
setuptools 49.6.0-py38h32f6830_0 --> 49.6.0-py36h9f0ad1d_0
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
> # activate that environment
> use_condaenv("loompy-test", required = TRUE)
> # is loompy available?
> py_module_available("loompy")
[1] FALSE
> |
|
Are you sure you're using the development version of reticulate? Did you restart R after installing the new package? |
I restarted the session (Terminate R from RStudio). I will try to uninstall the package and re-install. |
|
Ok I call this progress: Maybe there is something weird with my system? |
@micdonato were you able to figure it out? I have a similar error message: |
|
Nope, never was. I abandoned reticulate and rstudio and started using python packages directly from python.
It was a good run until it lasted.
Mic
… On Sep 24, 2021, at 12:05, Hyebin Han ***@***.***> wrote:
loompy <- reticulate::import('loompy')
Error in py_module_import(module, convert = convert) :
ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /local-scratch/users/myuser/miniconda3/envs/r-reticulate/lib/python3.6/site-packages/scipy/_lib/_uarray/_uarray.cpython-36m-x86_64-linux-gnu.so)
@micdonato were you able to figure it out? I have a similar error message:
Error in py_module_import(module, convert = convert) : SyntaxError: invalid syntax (files.py, line 191)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
|
|
😭😭 |
|
I want to clarify a couple of things though. The system I was trying to use reticulate on was our lab server. I suspect the issue was on the server side but we never figured out what it was. I was using RStudio Server Pro, and I believe I had no issues when I tried with the Desktop version on my work laptop. What was weird is that not even my sysadmin was able to figure out what happened. |
|
Interesting... I am having the opposite issue where I have issues with my desktop (I think it might be related to CPU) but not with our HPCC. Regardless, @micdonato , thank you for your response and input! Best of luck 🥲🤞🏼 |
I am having same problem when using reticulate on HPC server. |
|
This type of issue can also be due to reticulate selecting different conda installations when initializing and when installing. In a recent PR, system commands installing into a conda environment are now echoed, so it should be easier to debug and diagnose if this is the cause. Please try with the dev version of reticulate and see if that helps: remotes::install_github("rstudio/reticulate") |
Not working yet.
|
Problem solved: module load R |
|
The same problems |
|
reticulate can only found the python pakcage installed by system python binary with |
|
@Yunuuuu, can you please open a new issue with instructions for how to reproduce the error you're seeing? |
|
I thought my problem is the same with current issue and my test code is the same with this issue (I cannot use reprex with reticulate, I don't know why), here is my code examples: First try: options(reticulate.conda_binary = "/usr/local/bin/micromamba")
reticulate::conda_create("scanpy", "scanpy")
reticulate::import("scanpy")Here is the output: [R]> options(reticulate.conda_binary = "/usr/local/bin/micromamba")
[R]> reticulate::conda_create("scanpy", "scanpy")
+ /usr/local/bin/micromamba create --yes --name scanpy 'python=3.10' scanpy --quiet -c conda-forge
[1] "/home/pengyun/micromamba/envs/scanpy/bin/python"
[R]> reticulate::import("scanpy")
Error in py_module_import(module, convert = convert) :
ModuleNotFoundError: No module named 'scanpy'
Run `reticulate::py_last_error()` for details.
[R]>
Second try options(reticulate.conda_binary = "/usr/local/bin/micromamba")
reticulate::conda_create("scanpy", "scanpy")
# The requested version of Python cannot be used, as another version of
# Python ('/usr/bin/python3') has already been initialized.
Sys.setenv(RETICULATE_PYTHON = reticulate::conda_python("scanpy"))
reticulate::use_condaenv("scanpy")
reticulate::import("scanpy")here is the output [R]> options(reticulate.conda_binary = "/usr/local/bin/micromamba")
[R]> reticulate::conda_create("scanpy", "scanpy")
+ /usr/local/bin/micromamba create --yes --name scanpy 'python=3.10' scanpy --quiet -c conda-forge
[1] "/home/pengyun/micromamba/envs/scanpy/bin/python"
[R]> # The requested version of Python cannot be used, as another version of
# Python ('/usr/bin/python3') has already been initialized.
Sys.setenv(RETICULATE_PYTHON = reticulate::conda_python("scanpy"))
[R]> reticulate::use_condaenv("scanpy")
[R]> reticulate::import("scanpy")
Error in py_module_import(module, convert = convert) :
ModuleNotFoundError: No module named 'scanpy'
Run `reticulate::py_last_error()` for details.[R]> sessionInfo()
R version 4.2.3 (2023-03-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.2 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=zh_CN.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=zh_CN.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] progressr_0.14.0 Rcpp_1.0.12 lattice_0.20-45
[4] png_0.1-8 fansi_1.0.6 digest_0.6.35
[7] utf8_1.2.4 withr_3.0.0 grid_4.2.3
[10] lifecycle_1.0.4 jsonlite_1.8.8 pillar_1.9.0
[13] rlang_1.1.3 cli_3.6.2 Matrix_1.5-1
[16] vctrs_0.6.5 reticulate_1.37.0.9000 tools_4.2.3
[19] glue_1.7.0 compiler_4.2.3 |
|
But I can fix this by specifying the environment in import: reticulate::import("scanpy", delay_load = list(environment = "scanpy")) |
|
The "Second try" approach is the one I would expect to work, except it seems that reticulate had already initialized another Python when you ran it. Can you please try it in a fresh R session? |
|
It's the same: [R]> options(reticulate.conda_binary = "/usr/local/bin/micromamba")
[R]> reticulate::conda_create("scanpy", "scanpy")
+ /usr/local/bin/micromamba create --yes --name scanpy 'python=3.10' scanpy --quiet -c conda-forge
[1] "/home/pengyun/micromamba/envs/scanpy/bin/python"
[R]> # The requested version of Python cannot be used, as another version of
# Python ('/usr/bin/python3') has already been initialized.
Sys.setenv(RETICULATE_PYTHON = reticulate::conda_python("scanpy"))
[R]> reticulate::use_condaenv("scanpy")
[R]> reticulate::import("scanpy")
Error in py_module_import(module, convert = convert) :
ModuleNotFoundError: No module named 'scanpy'
Run `reticulate::py_last_error()` for details.
|
The error message is very useful here. It is telling you that something has already initiailzed Python. You must figure out why, and disable it. Most likely it is something in |
Hi all.
I am trying to install
loompyandanndatafor use withsceasy.This is my output, from a fresh session. The module installs properly, it seems, but
reticulatecannot detect it.Strangely enough, this does not happen for
numpy, but only forloompyThis is the code I use:
Here I am setting the empty environment variables
PYTHONHOMEandPYTHONPATHbut nothing changes if I remove them.Also, I am forcing the use of
r-reticulateas an environment, but again, nothing changes if I remove that.This is the output:
What confuses me is that everything seems to be where it needs to be: the conda and python binaries, the environment.
I checked the python directory:
ls /local-scratch/users/myuser/miniconda3/envs/r-reticulate/lib/python3.6/site-packages/and this is the result:
Now, the only thing I see different is the following:
py_configseems to return a "python 3.8.5" version, whileconda_install()installs the package for 3.6.11.So I tried to install python first on
r-reticulate, in order to have it updated to the latest version, but then loompy downgrades it and I am back to square zero...The fun thing is that with a combination of lines of code I was able to get a
py_configwith 3.6.11, but then againloompyis not available. If I try to install it, guess what? It upgrades python to 3.8.5.In other words, I seem not to be ever able to use loompy that is installed in the directory:
/local-scratch/users/myuser/miniconda3/envs/r-reticulate/lib/python3.6/site-packages/I am going crazy.
The text was updated successfully, but these errors were encountered: