ERROR: Could not build wheels for hdbscan which use PEP 517 and cannot be installed directly

[boompig]

python: Python 2.7.15rc1

OS: Linux 6a039c3530c7 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

pip: pip 19.1

tried running: pip install hdbscan

  Building wheel for hdbscan (PEP 517): finished with status 'error'                                                   │
  ERROR: Complete output from command /opt/conda/bin/python /opt/conda/lib/python3.6/site-packages/pip/_vendor/pep517/_│
in_process.py build_wheel /tmp/tmpvwnr9hhz:                                                                            │
  ERROR: running bdist_wheel                                                                                           │
  running build                                                                                                        │
  running build_py                                                                                                     │
  creating build                                                                                                       │
  creating build/lib.linux-x86_64-3.6                                                                                  │
  creating build/lib.linux-x86_64-3.6/hdbscan                                                                          │
  copying hdbscan/prediction.py -> build/lib.linux-x86_64-3.6/hdbscan                                                  │
  copying hdbscan/robust_single_linkage_.py -> build/lib.linux-x86_64-3.6/hdbscan                                      │
  copying hdbscan/__init__.py -> build/lib.linux-x86_64-3.6/hdbscan                                                    │
  copying hdbscan/validity.py -> build/lib.linux-x86_64-3.6/hdbscan                                                    │
  copying hdbscan/plots.py -> build/lib.linux-x86_64-3.6/hdbscan                                                       │
  copying hdbscan/hdbscan_.py -> build/lib.linux-x86_64-3.6/hdbscan                                                    │
  creating build/lib.linux-x86_64-3.6/hdbscan/tests                                                                    │
  copying hdbscan/tests/test_rsl.py -> build/lib.linux-x86_64-3.6/hdbscan/tests                                        │
  copying hdbscan/tests/__init__.py -> build/lib.linux-x86_64-3.6/hdbscan/tests                                        │
  copying hdbscan/tests/test_hdbscan.py -> build/lib.linux-x86_64-3.6/hdbscan/tests                                    │
  running build_ext                                                                                                    │
  cythoning hdbscan/_hdbscan_tree.pyx to hdbscan/_hdbscan_tree.c                                                       │
  building 'hdbscan._hdbscan_tree' extension                                                                           │
  creating build/temp.linux-x86_64-3.6                                                                                 │
  creating build/temp.linux-x86_64-3.6/hdbscan                                                                         │
  gcc -pthread -B /opt/conda/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prot│
otypes -fPIC -I/opt/conda/include/python3.6m -I/tmp/pip-build-env-s09rgxp0/overlay/lib/python3.6/site-packages/numpy/co│
re/include -c hdbscan/_hdbscan_tree.c -o build/temp.linux-x86_64-3.6/hdbscan/_hdbscan_tree.o                           │
  /tmp/pip-build-env-s09rgxp0/overlay/lib/python3.6/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython di│
rective 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-rf│
yrnh0q/hdbscan/hdbscan/_hdbscan_tree.pyx                                                                               │
    tree = Parsing.p_module(s, pxd, full_module_name)                                                                  │
  error: command 'gcc' failed with exit status 1                                                                       │
  ----------------------------------------                                                                             │
  ERROR: Failed building wheel for hdbscan                                                                             │
  Running setup.py clean for hdbscan                                                                                   │
Failed to build hdbscan                                                                                                │
ERROR: Could not build wheels for hdbscan which use PEP 517 and cannot be installed directly                           │
The command '/bin/sh -c pip install hdbscan' returned a non-zero code: 1                                               │```

[boompig]

sorry gcc wasn't installed in the container and pip was not reporting the error

[flycloudking]

Hi,

I got the same error, i didn't understand what's the issue. I do have gcc installed.

Thanks

[lmcinnes]

I'm not sure what your system setup is, but gcc can be installed but if python can't find it (i.e. it isn't on the PATH or something similar) then it can still fail. Do other C or Cython based python packages build successfully?

[javierorozco]

I am getting the same error in Databricks, they use ubuntu ec2 instances running python 3.5 and pip 19.1.

[chaowu2009]

ERROR: Could not build wheels for hdbscan which use PEP 517 and cannot be installed directly
MacOS Majove, 10.14.3

[addiwei]

I would not advise using pip install to install anything in Databricks. I am trying to figure out how to install hdbscan via "Library" which should be the proper way to install.

I am getting the same error in Databricks, they use ubuntu ec2 instances running python 3.5 and pip 19.1.

[ntnshrav]

The issue was with python-dev not installed.
Once you install python-dev tools the package HDBSCAN installed successfully.
command: sudo apt-get install python3.6-dev .

[niccolomenicori]

I have this problem with PEP517 installing pupil-detectors. Do you know how can I solve this problem?

[gaalocastillo]

I am having this problem using Windows and Anaconda3. How have you solved this problem having those condtions?

[yashchoubey]

sudo apt-get install python3.7-dev worked for me

[Shashank-Gupta-102]

Please try installing Microsoft Visual C++ 14.0. It worked for me. Or well, that was missing and after that hdbscan library installed quickly

[yinglinglow]

If you are using anaconda/miniconda, you can use conda install -c conda-forge hdbscan - it worked for me (I was installing it in a Docker container).

[shriaas2898]

sudo apt-get install python3.7-dev worked for me

I tried installing python dev tools still it is not able to build wheels

OS: Fedora 30
Python 3.7

[Zwartfreak]

conda install -c conda-forge hdbscan

For Anaconda users.

[marcmaxson]

When installing hdbscan using a Dockerfile you cannot use python:3.7-slim-stretch or python:3.7-alpine but FROM python3.7 works:

FROM python:3.7
#FROM python:3.7.7-slim-stretch --- does NOT work with hdbscan (missing gcc or cython or something)
WORKDIR /app
COPY source/requirements.txt ./
RUN set -ex && pip3 install -r requirements.txt
COPY source /app
ENTRYPOINT  ["python"]
CMD [ "main.py" ]

[ArjunSarihyan]

e hdbscan

This worked
Thanks

[ScDor]

On windows 10, solved by installing directly from a precompiled wheel file, find it here

[SirHodlington]

I'm using Windows 10 and had this problem initially. Fixed by installing the C++ Desktop Development package inside the Visual Studio Community Installer.

[hubgitadi]

I was facing a similar kind of issue (not related to gcc), installation directly from the Github repo worked
https://github.com/scikit-learn-contrib/hdbscan
pip install --upgrade git+https://github.com/scikit-learn-contrib/hdbscan.git#egg=hdbscan

from .mtrand import RandomState
File "mtrand.pyx", line 1, in init numpy.random.mtrand
File "bit_generator.pyx", line 1, in init numpy.random.bit_generator
ModuleNotFoundError: No module named 'numpy.random._common'

ERROR: Failed building wheel for hdbscan
Failed to build hdbscan
ERROR: Could not build wheels for hdbscan which use PEP 517 and cannot be installed directly

[harshgeek4coder]

I was using LInux based Environment with Ubuntu 20.0.4 OS. For me, the following steps worked :

• sudo apt-get install gcc
• sudo apt-get install build-essential -y

After installing the above two, I did pip install hdbscan, and it worked.

[dtanman]

Just to update, the suggestion of @harshgeek4coder was what helped in my case; I didn't want to use conda just for this.

[harshgeek4coder]

Just to update, the suggestion of @harshgeek4coder was what helped in my case; I didn't want to use conda just for this.

Actually, conda forge install was working totally fine, I tested it out with a different environments too.
It's just that while using Linux-based instance or bash-based terminals for a full clean install of hdbscan, conda is not always available, hence, I worked with the before mentioned steps here and it worked fine both on Windows and Linux.

[AntaresRahman]

On Arch, solved issue by running sudo pacman -S base-devel

https://bbs.archlinux.org/viewtopic.php?id=39935

[Akshayextreme]

I was using LInux based Environment with Ubuntu 20.0.4 OS. For me, the following steps worked :

• sudo apt-get install gcc
• sudo apt-get install build-essential -y

After installing the above two, I did pip install hdbscan, and it worked.

For me this did not work. But below solution did work
sudo apt install python3.8-dev

Reference link

[parshin76]

On windows 10, solved by installing directly from a precompiled wheel file, find it here

This worked for me. I'm using Windows 10. Python 3.10. Many thanks!

[phuclh]

If you are using anaconda/miniconda, you can use conda install -c conda-forge hdbscan - it worked for me (I was installing it in a Docker container).

Hi @yinglinglow, how did you install conda in Docker container? Can I have your Dockerfile, thank you!

[JohnMagnusson]

I got python 3.8.10 and sudo apt-get install python3.8-dev worked for me solved my issue