Missing pkg_resources attribute from umap-learn (v 0.5.5)

[fspecque]

Hi !

I've noticed a bug when computing UMAP using a Graph (thus, using umap-learn). See the code below to reproduce. Please note that the version of the umap-learn python module I'm using is 0.5.5 and I've installed it using conda (conda-forge)

library(Seurat)

data(pbmc_small)

pbmc_small <- FindVariableFeatures(pbmc_small, verbose = F)
pbmc_small <- ScaleData(pbmc_small, verbose = F)
pbmc_small <- RunPCA(pbmc_small, verbose = F)
pbmc_small <- FindNeighbors(pbmc_small, verbose = F)
pbmc_small <- RunUMAP(pbmc_small, graph = "RNA_snn", umap.method = "umap-learn")

# Error in py_get_attr_impl(x, name, silent) : 
#   AttributeError: module 'umap' has no attribute 'pkg_resources'
# Run `reticulate::py_last_error()` for details.

Here's the output of reticulate::py_last_error()

# ── Python Exception Message ───────────────────────────────────────────────────────────────────────
# AttributeError: module 'umap' has no attribute 'pkg_resources'

# ── R Traceback ────────────────────────────────────────────────────────────────────────────────────
#      ▆
#   1. ├─Seurat::RunUMAP(pbmc_small, graph = "RNA_snn", umap.method = "umap-learn")
#   2. └─Seurat:::RunUMAP.Seurat(pbmc_small, graph = "RNA_snn", umap.method = "umap-learn")
#   3.   ├─Seurat::RunUMAP(...)
#   4.   └─Seurat:::RunUMAP.Graph(...)
#   5.     ├─base::numeric_version(x = umap$pkg_resources$get_distribution("umap-learn")$version)
#   6.     │ └─base::.make_numeric_version(x, strict, .standard_regexps()$valid_numeric_version)
#   7.     ├─umap$pkg_resources
#   8.     └─reticulate:::`$.python.builtin.module`(umap, "pkg_resources")
#   9.       └─reticulate:::`$.python.builtin.object`(x, name)
#  10.         └─reticulate:::py_get_attr_or_item(x, name, TRUE)
#  11.           └─reticulate::py_get_attr(x, name)
#  12.             └─reticulate:::py_get_attr_impl(x, name, silent)

I've noticed in seurat/R/dimensional_reduction.R that the way to check for the version of umap-learn is not compatible with version 0.5.5 :

umap <- reticulate::import("umap", delay_load = TRUE)
umap$pkg_resources$get_distribution("umap-learn")$version
# Error in py_get_attr_impl(x, name, silent) : 
# AttributeError: module 'umap' has no attribute 'pkg_resources'
# Run `reticulate::py_last_error()` for details.

# But this works
umap$version("umap-learn")
# [1] "0.5.5"

It's seems that the change happen for version 0.5.5 (lmcinnes/umap@c427b48)

Thus, I think there might be a way to correct that, maybe with a tryCatch ? Also, umap$`__version__` might be a valid alternative.
Best,
Florian

PS: Here is the output of sessionInfo()

R version 4.3.2 (2023-10-31)
Platform: x86_64-conda-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS/LAPACK: /gpfs/scratch/CBIB/fspecque/miniconda3/envs/scReticulate/lib/libopenblasp-r0.3.25.so;  LAPACK version 3.11.0

locale:
 [1] LC_CTYPE=fr_FR.UTF-8       LC_NUMERIC=C               LC_TIME=fr_FR.UTF-8        LC_COLLATE=fr_FR.UTF-8     LC_MONETARY=fr_FR.UTF-8    LC_MESSAGES=fr_FR.UTF-8    LC_PAPER=fr_FR.UTF-8      
 [8] LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=fr_FR.UTF-8 LC_IDENTIFICATION=C       

time zone: Europe/Paris
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] Seurat_5.0.1       SeuratObject_5.0.1 sp_2.1-2          

loaded via a namespace (and not attached):
  [1] deldir_2.0-2           pbapply_1.7-2          gridExtra_2.3          rlang_1.1.2            magrittr_2.0.3         RcppAnnoy_0.0.21       matrixStats_1.2.0      ggridges_0.5.4        
  [9] compiler_4.3.2         spatstat.geom_3.2-7    png_0.1-8              vctrs_0.6.5            reshape2_1.4.4         stringr_1.5.1          pkgconfig_2.0.3        fastmap_1.1.1         
 [17] ellipsis_0.3.2         utf8_1.2.4             promises_1.2.1         purrr_1.0.2            jsonlite_1.8.8         goftest_1.2-3          later_1.3.2            spatstat.utils_3.0-4  
 [25] irlba_2.3.5.1          parallel_4.3.2         cluster_2.1.6          R6_2.5.1               ica_1.0-3              stringi_1.8.3          RColorBrewer_1.1-3     spatstat.data_3.0-3   
 [33] reticulate_1.34.0      parallelly_1.36.0      lmtest_0.9-40          scattermore_1.2        Rcpp_1.0.11            tensor_1.5             future.apply_1.11.0    zoo_1.8-12            
 [41] sctransform_0.4.1      httpuv_1.6.11          Matrix_1.6-4           splines_4.3.2          igraph_1.6.0           tidyselect_1.2.0       abind_1.4-5            spatstat.random_3.2-2 
 [49] codetools_0.2-19       miniUI_0.1.1.1         spatstat.explore_3.2-5 listenv_0.9.0          lattice_0.22-5         tibble_3.2.1           plyr_1.8.9             withr_2.5.2           
 [57] shiny_1.8.0            ROCR_1.0-11            Rtsne_0.17             future_1.33.0          fastDummies_1.7.3      survival_3.5-7         polyclip_1.10-6        fitdistrplus_1.1-11   
 [65] pillar_1.9.0           KernSmooth_2.23-22     plotly_4.10.3          generics_0.1.3         RcppHNSW_0.5.0         ggplot2_3.4.4          munsell_0.5.0          scales_1.3.0          
 [73] globals_0.16.2         xtable_1.8-4           glue_1.6.2             lazyeval_0.2.2         tools_4.3.2            data.table_1.14.10     RSpectra_0.16-1        RANN_2.6.1            
 [81] leiden_0.4.3.1         dotCall64_1.1-1        cowplot_1.1.1          grid_4.3.2             tidyr_1.3.0            colorspace_2.1-0       nlme_3.1-164           patchwork_1.1.3       
 [89] cli_3.6.2              spatstat.sparse_3.0-3  spam_2.10-0            fansi_1.0.6            viridisLite_0.4.2      dplyr_1.1.4            uwot_0.1.16            gtable_0.3.4          
 [97] digest_0.6.33          progressr_0.14.0       ggrepel_0.9.4          htmlwidgets_1.6.4      htmltools_0.5.7        lifecycle_1.0.4        httr_1.4.7             mime_0.12             
[105] MASS_7.3-60 

[MoreiZH]

I had the same issue with the same seurat5.01 and umap-learn0.5.5 installed via Conda.

[RijndertAriese]

I had the same error and solved it for now by downgrading my umap-learn to 0.5.3 via reticulate.

[jestlin15]

Hi, I tried with umap-learn v0.5.3 but I end up with another error : 0it [00:00, ?it/s]

How may I be able to solve this please?

Here is my codes:

scdata_integrated <- RunUMAP(scdata_integrated, reduction = "pca",
+                              umap.method = "umap-learn",
+                              graph = "integrated_snn")

Also attaching my session info:

R version 4.3.1 (2023-06-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.3 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so;  LAPACK version 3.10.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: Asia/Seoul
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] cowplot_1.1.1      viridis_0.6.4      viridisLite_0.4.2  ggplot2_3.4.4      dplyr_1.1.4        SeuratObject_4.1.4 Seurat_4.3.0       reticulate_1.34.0 

loaded via a namespace (and not attached):
  [1] deldir_2.0-2           pbapply_1.7-2          gridExtra_2.3          rlang_1.1.2            magrittr_2.0.3         RcppAnnoy_0.0.21       matrixStats_1.1.0     
  [8] ggridges_0.5.4         compiler_4.3.1         spatstat.geom_3.2-7    png_0.1-8              vctrs_0.6.5            reshape2_1.4.4         stringr_1.5.1         
 [15] pkgconfig_2.0.3        fastmap_1.1.1          ellipsis_0.3.2         labeling_0.4.3         utf8_1.2.4             promises_1.2.1         purrr_1.0.2           
 [22] xfun_0.41              jsonlite_1.8.7         goftest_1.2-3          later_1.3.1            spatstat.utils_3.0-4   irlba_2.3.5.1          parallel_4.3.1        
 [29] cluster_2.1.4          R6_2.5.1               ica_1.0-3              stringi_1.8.2          RColorBrewer_1.1-3     spatstat.data_3.0-3    parallelly_1.36.0     
 [36] lmtest_0.9-40          scattermore_1.2        knitr_1.45             Rcpp_1.0.11            tensor_1.5             future.apply_1.11.0    zoo_1.8-12            
 [43] sctransform_0.4.1      httpuv_1.6.12          Matrix_1.6-1           splines_4.3.1          igraph_1.5.1           tidyselect_1.2.0       rstudioapi_0.15.0     
 [50] abind_1.4-5            spatstat.random_3.2-2  codetools_0.2-19       miniUI_0.1.1.1         spatstat.explore_3.2-5 listenv_0.9.0          lattice_0.21-8        
 [57] tibble_3.2.1           plyr_1.8.9             withr_2.5.2            shiny_1.8.0            ROCR_1.0-11            Rtsne_0.16             future_1.33.0         
 [64] survival_3.5-7         polyclip_1.10-6        fitdistrplus_1.1-11    pillar_1.9.0           KernSmooth_2.23-22     plotly_4.10.3          generics_0.1.3        
 [71] sp_2.1-2               munsell_0.5.0          scales_1.3.0           globals_0.16.2         xtable_1.8-4           glue_1.6.2             lazyeval_0.2.2        
 [78] tools_4.3.1            data.table_1.14.8      RANN_2.6.1             leiden_0.4.3.1         grid_4.3.1             tidyr_1.3.0            colorspace_2.1-0      
 [85] nlme_3.1-163           patchwork_1.1.3        cli_3.6.1              spatstat.sparse_3.0-3  fansi_1.0.5            uwot_0.1.16            gtable_0.3.4          
 [92] digest_0.6.33          progressr_0.14.0       ggrepel_0.9.4          farver_2.1.1           htmlwidgets_1.6.3      htmltools_0.5.7        lifecycle_1.0.4       
 [99] httr_1.4.7             mime_0.12              MASS_7.3-60 

[z5ouyang]

have to downgrade the umap-learn=0.5.4 in conda env

[kshakir]

Thanks for the tips on downgrading umap-learn. Hopefully, the PR from January listed above will be reviewed soon. 🤞

Also a reminder to folks to restart their R sessions after downgrading. Seurat uses reticulate which loads python imports into memory.

You can double-check the versions of loaded libraries via:

> cat(names(reticulate::import("umap")), sep="\n")
aligned_umap
AlignedUMAP
catch_warnings
distances
layouts
numba
ParametricUMAP
pkg_resources
simplefilter
sparse
spectral
UMAP
umap_
utils
warn
> subset(reticulate::py_list_packages(), package=="umap-learn")
      package version      requirement channel
24 umap-learn   0.5.4 umap-learn=0.5.4    pypi
> umap_import$`__version__`
[1] "0.5.4"
> packageVersion("Seurat")
[1] ‘5.0.1’
> reticulate::py_config()
[elided]
umap:           [elided]/lib/python3.11/site-packages/umap

NOTE: Python version was forced by use_python() function
> 

If you've downgraded umap to 0.5.4 in the path returned by reticulate::py_config() using pip, conda, etc., and the version of umap is still a new version without a pkg_resources, restart your R session and try these version checks again.

[nathanweeks]

When installing umap-learn=0.5.4 with reticulate::py_install(), there is now a problem with numpy (which recently had a major 2.0 release):

> library(Seurat)
...
> data("pbmc_small")
> pbmc_small <- RunUMAP(object = pbmc_small, dims = 1:5, metric='correlation', umap.method='umap-learn')
Error: Required version of NumPy not available: incompatible NumPy binary version 33554432 (expecting version 16777225)
> reticulate::py_list_packages()
         package   version          requirement
1         joblib     1.4.2        joblib==1.4.2
2       llvmlite    0.43.0     llvmlite==0.43.0
3          numba    0.60.0        numba==0.60.0
4          numpy     2.0.0         numpy==2.0.0
5    pynndescent    0.5.13  pynndescent==0.5.13
6   scikit-learn     1.5.1  scikit-learn==1.5.1
7          scipy    1.14.0        scipy==1.14.0
8            tbb 2021.13.0       tbb==2021.13.0
9  threadpoolctl     3.5.0 threadpoolctl==3.5.0
10          tqdm    4.66.4         tqdm==4.66.4
11    umap-learn     0.5.4    umap-learn==0.5.4

Additionally specifying a compatible numpy version seems to work around the issue:

reticulate::py_install(packages = c("umap-learn==0.5.4","numpy<2"))

[methornton]

Hello!

I messed up and accidental upgraded "conda upgrade all". I have downgraded, but I'm still getting this error:

Error in py_get_attr(x, name, FALSE) : 
  AttributeError: module 'umap' has no attribute 'pkg_resources'
Run `reticulate::py_last_error()` for details.
Calls: %>% ... is.numeric_version -> $ -> $.python.builtin.module -> py_get_attr

Here are my checks. I reinstalled conda using the python for the version of Ubuntu I'm using is 3.8.10, should I upgrade it?

> library(reticulate)
> cat(names(reticulate::import("umap")), sep="\n")
aligned_umap
AlignedUMAP
catch_warnings
distances
layouts
numba
PackageNotFoundError ## Is there where pkg_resources should be?
ParametricUMAP
simplefilter
sparse
spectral
UMAP
umap_
utils
version
warn
> subset(reticulate::py_list_packages(), package=="umap-learn")
      package version      requirement   channel
50 umap-learn   0.5.4 umap-learn=0.5.4 pkgs/main
> umap_import$`__version__`
Error: object 'umap_import' not found
> packageVersion("Seurat")
[1] ‘5.1.0’
> reticulate::py_config()
python:         /opt/anaconda3/envs/r-reticulate/bin/python
libpython:      /opt/anaconda3/envs/r-reticulate/lib/libpython3.8.so
pythonhome:     /opt/anaconda3/envs/r-reticulate:/opt/anaconda3/envs/r-reticulate
version:        3.8.10 (default, Jun  4 2021, 15:09:15)  [GCC 7.5.0]
numpy:          /home/met/.local/lib/python3.8/site-packages/numpy
numpy_version:  1.24.4
umap:           /home/met/.local/lib/python3.8/site-packages/umap
> sessionInfo()
R version 4.4.1 (2024-06-14)
Platform: x86_64-pc-linux-gnu
Running under: Ubuntu 20.04.6 LTS

Matrix products: default
BLAS/LAPACK: /usr/local/lib/libopenblas_zenp-r0.3.18.so;  LAPACK version 3.9.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: America/Los_Angeles
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] reticulate_1.38.0

loaded via a namespace (and not attached):
 [1] compiler_4.4.1  here_1.0.1      Matrix_1.7-0    rprojroot_2.0.4
 [5] cli_3.6.3       withr_3.0.1     rappdirs_0.3.3  Rcpp_1.0.13    
 [9] grid_4.4.1      jsonlite_1.8.8  rlang_1.1.4     png_0.1-8      
[13] lattice_0.22-6 

I'm kind of stuck as I'm in the middle of a large dataset and I need these to compare to the others. I guess this has been open for a long while. Any advice is greatly appreciated.