Error when running UMAP

[rjesud]

Hi, Great package! I am trying to run PARC's UMAP implementation but I receive the following error:

It appears that this issue has popped up here: scverse/scanpy#1579

I tried stepping back to umap-learn==0.4.5 but I encounter an error when importing PARC.

Any help would be greatly appreciated.

My venv:

_libgcc_mutex 0.1 main
backcall 0.2.0
blas 1.0 mkl
ca-certificates 2021.5.25 h06a4308_1
certifi 2021.5.30 py37h06a4308_0
cycler 0.10.0 py37_0
dbus 1.13.18 hb2f20db_0
decorator 5.0.9
expat 2.4.1 h2531618_2
fontconfig 2.13.1 h6c09931_0
freetype 2.10.4 h5ab3b9f_0
glib 2.68.2 h36276a3_0
gst-plugins-base 1.14.0 h8213a91_2
gstreamer 1.14.0 h28cd5cc_2
hnswlib 0.5.1
icu 58.2 he6710b0_3
intel-openmp 2021.2.0 h06a4308_610
ipykernel 5.5.5
ipython 7.24.1
ipython-genutils 0.2.0
jedi 0.18.0
joblib 1.0.1 pyhd3eb1b0_0
jpeg 9b h024ee3a_2
jupyter-client 6.1.12
jupyter-core 4.7.1
kiwisolver 1.3.1 py37h2531618_0
lcms2 2.12 h3be6417_0
ld_impl_linux-64 2.33.1 h53a641e_7
leidenalg 0.8.4
libffi 3.3 he6710b0_2
libgcc-ng 9.1.0 hdf63c60_0
libgfortran-ng 7.3.0 hdf63c60_0
libpng 1.6.37 hbc83047_0
libstdcxx-ng 9.1.0 hdf63c60_0
libtiff 4.2.0 h85742a9_0
libuuid 1.0.3 h1bed415_2
libwebp-base 1.2.0 h27cfd23_0
libxcb 1.14 h7b6447c_0
libxml2 2.9.10 hb55368b_3
llvmlite 0.36.0
lz4-c 1.9.3 h2531618_0
matplotlib 3.3.4 py37h06a4308_0
matplotlib-base 3.3.4 py37h62a2d02_0
matplotlib-inline 0.1.2
mkl 2021.2.0 h06a4308_296
mkl-service 2.3.0 py37h27cfd23_1
mkl_fft 1.3.0 py37h42c9631_2
mkl_random 1.2.1 py37ha9443f7_2
ncurses 6.2 he6710b0_1
numba 0.53.1
numpy 1.20.2 py37h2d18471_0
numpy-base 1.20.2 py37hfae3a4d_0
libstdcxx-ng 9.1.0 hdf63c60_0
olefile 0.46 py_0
openssl 1.1.1k h27cfd23_0
pandas 1.2.4 py37h2531618_0
parc 0.31
parso 0.8.2
pcre 8.44 he6710b0_0
pexpect 4.8.0
pickleshare 0.7.5
pillow 8.2.0 py37he98fc37_0
pip 21.1.1 py37h06a4308_0
prompt-toolkit 3.0.18
ptyprocess 0.7.0
pybind11 2.6.2 py37hff7bd54_1
Pygments 2.9.0
pynndescent 0.5.2
pyparsing 2.4.7 pyhd3eb1b0_0
pyqt 5.9.2 py37h05f1152_2
python 3.7.10 hdb3f193_0
python-dateutil 2.8.1 pyhd3eb1b0_0
python-igraph 0.9.4
pytz 2021.1 pyhd3eb1b0_0
pyzmq 22.1.0
qt 5.9.7 h5867ecd_1
readline 8.1 h27cfd23_0
scikit-learn 0.24.2 py37ha9443f7_0
scipy 1.6.2 py37had2a1c9_1
setuptools 52.0.0 py37h06a4308_0
sip 4.19.8 py37hf484d3e_0
six 1.15.0 pyhd3eb1b0_0
sqlite 3.35.4 hdfb4753_0
texttable 1.6.3
threadpoolctl 2.1.0 pyh5ca1d4c_0
tk 8.6.10 hbc83047_0
tornado 6.1 py37h27cfd23_0
traitlets 5.0.5
umap-learn 0.5.1
wcwidth 0.2.5
wheel 0.36.2 pyhd3eb1b0_0
xz 5.2.5 h7b6447c_0
zlib 1.2.11 h7b6447c_3
zstd 1.4.9 haebb681_0

[ShobiStassen]

Hi, i have the following suggestion until i get around to doing a proper fix.
I just installed and ran PARC with the hybrid umap construction on a clean environment and get it to work by ensuring that 1) numba is version 0.49.1 and 2) umap-learn is 0.4.3

import parc
import matplotlib.pyplot as plt

from sklearn import datasets

iris = datasets.load_iris()
X = iris.data
y = iris.target

p1 = parc.PARC(X, true_label=y, too_big_factor=0.3, resolution_parameter=1, keep_all_local_dist=True)  
p1.run_PARC()
print(type(p1.labels), p1.stats_df)

graph = p1.knngraph_full()
X_umap = p1.run_umap_hnsw(X, graph)
plt.scatter(X_umap[:, 0], X_umap[:, 1], c=p1.labels)
plt.show()
plt.scatter(X_umap[:, 0], X_umap[:, 1], c=y)
plt.show()

(ParcEnv2021) shobi@shobi:~$ conda list
packages in environment at /home/shobi/anaconda3/envs/ParcEnv2021:

Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 1_gnu conda-forge
ca-certificates 2021.5.30 ha878542_0 conda-forge
certifi 2021.5.30 py37h89c1867_0 conda-forge
cycler 0.10.0 pypi_0 pypi
hnswlib 0.5.1 pypi_0 pypi
joblib 1.0.1 pypi_0 pypi
kiwisolver 1.3.1 pypi_0 pypi
ld_impl_linux-64 2.35.1 hea4e1c9_2 conda-forge
leidenalg 0.7.0 pypi_0 pypi
libffi 3.3 h58526e2_2 conda-forge
libgcc-ng 9.3.0 h2828fa1_19 conda-forge
libgomp 9.3.0 h2828fa1_19 conda-forge
libstdcxx-ng 9.3.0 h6de172a_19 conda-forge
llvmlite 0.32.1 pypi_0 pypi
matplotlib 3.4.2 pypi_0 pypi
ncurses 6.2 h58526e2_4 conda-forge
numba 0.49.1 pypi_0 pypi
numpy 1.20.3 pypi_0 pypi
openssl 1.1.1k h7f98852_0 conda-forge
pandas 1.2.4 pypi_0 pypi
parc 0.31 pypi_0 pypi
pillow 8.2.0 pypi_0 pypi
pip 21.1.2 pyhd8ed1ab_0 conda-forge
pybind11 2.6.2 pypi_0 pypi
pyparsing 2.4.7 pypi_0 pypi
python 3.7.10 hffdb5ce_100_cpython conda-forge
python-dateutil 2.8.1 pypi_0 pypi
python-igraph 0.9.4 pypi_0 pypi
python_abi 3.7 1_cp37m conda-forge
pytz 2021.1 pypi_0 pypi
readline 8.1 h46c0cb4_0 conda-forge
scikit-learn 0.24.2 pypi_0 pypi
scipy 1.6.3 pypi_0 pypi
setuptools 49.6.0 py37h89c1867_3 conda-forge
six 1.16.0 pypi_0 pypi
sqlite 3.35.5 h74cdb3f_0 conda-forge
tbb 2021.2.0 pypi_0 pypi
texttable 1.6.3 pypi_0 pypi
threadpoolctl 2.1.0 pypi_0 pypi
tk 8.6.10 h21135ba_1 conda-forge
umap-learn 0.4.3 pypi_0 pypi
wheel 0.36.2 pyhd3deb0d_0 conda-forge
xz 5.2.5 h516909a_1 conda-forge
zlib 1.2.11 h516909a_1010 conda-forge

[ShobiStassen]

These are the commands in a new venv :
conda create --ParcEnv2021 python=3.7
conda install pip
pip install matplotlib
pip install umap-learn==0.4.3 
pip install numba==0.49.1
pip install leidenalg #0.7.0 is faster so you could consider using 0.7.0 version for large data
pip install hnswlib
pip install python-igraph
pip install parc

[rjesud]

Thank you, @ShobiStassen! All is well now.

I received an compilation error when for "pip install leidanalg==0.7.0" but "pip install leidanalg" worked fine. It gave me 0.8.4. I did see your inquiry here: vtraag/leidenalg#35

It seems they fixed the slow down issue??