Unable to pip install crewai due to bug with compile chroma-hnswlib / Windows 11 latest WSL2 (TLDR: Old chroma db version)

[codeninja]

I am running into some significant issues installing the latest version using pip install on an Windows + WSL2 ubuntu 22.04.2 LTS

  Building wheel for chroma-hnswlib (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for chroma-hnswlib (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [21 lines of output]
      running bdist_wheel
      running build
      running build_ext
      creating tmp
      x86_64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/home/codeninja/swooner/.venv/include -I/usr/include/python3.12 -c /tmp/tmpn4nv0p1t.cpp -o tmp/tmpn4nv0p1t.o -std=c++14
      x86_64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/home/codeninja/swooner/.venv/include -I/usr/include/python3.12 -c /tmp/tmps66hcz8q.cpp -o tmp/tmps66hcz8q.o -fvisibility=hidden
      building 'hnswlib' extension
      creating build
      creating build/temp.linux-x86_64-cpython-312
      creating build/temp.linux-x86_64-cpython-312/python_bindings
      x86_64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/tmp/pip-build-env-myzn5zze/overlay/lib/python3.12/site-packages/pybind11/include -I/tmp/pip-build-env-myzn5zze/overlay/lib/python3.12/site-packages/numpy/_core/include -I./hnswlib/ -I/home/codeninja/swooner/.venv/include -I/usr/include/python3.12 -c ./python_bindings/bindings.cpp -o build/temp.linux-x86_64-cpython-312/./python_bindings/bindings.o -O3 -march=native -fopenmp -DVERSION_INFO=\"0.7.3\" -std=c++14 -fvisibility=hidden
      In file included from /tmp/pip-build-env-myzn5zze/overlay/lib/python3.12/site-packages/pybind11/include/pybind11/detail/../attr.h:13,
                       from /tmp/pip-build-env-myzn5zze/overlay/lib/python3.12/site-packages/pybind11/include/pybind11/detail/class.h:12,
                       from /tmp/pip-build-env-myzn5zze/overlay/lib/python3.12/site-packages/pybind11/include/pybind11/pybind11.h:13,
                       from /tmp/pip-build-env-myzn5zze/overlay/lib/python3.12/site-packages/pybind11/include/pybind11/functional.h:12,
                       from ./python_bindings/bindings.cpp:2:
      /tmp/pip-build-env-myzn5zze/overlay/lib/python3.12/site-packages/pybind11/include/pybind11/detail/../detail/common.h:274:10: fatal error: Python.h: No such file or directory
        274 | #include <Python.h>
            |          ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for chroma-hnswlib
Failed to build chroma-hnswlib
ERROR: Could not build wheels for chroma-hnswlib, which is required to install pyproject.toml-based projects```

This is likely a compile issue but I've been able to get no where with it.

[codeninja]

If I manually install the library it installs and compiles...

I did a fresh reboot of my compiler tools on Python=3.12.4. So I know my compiler is properly connected.

[codeninja]

Looking into the poetry lock it looks like you are a couple of versions behind their LTS. You have 0.7.3 vs current 0.7.5.

[codeninja]

And it looks like 0.7.3 is the culprit.

[codeninja]

And that looks to be the dependency of ChromaDB sitting at version 0.4.24 with the latest version a full major version ahead (0.5.3).

So yeah if yall could just update your foundational vector layer for me... that'd be great. /s

I'm not sure how to proceed at this point. Any help would be appreciated.

[theCyberTech]

What version of crewai are you using?

[codeninja]

Latest

…

On Wed, Jul 10, 2024, 12:33 AM Rip&Tear ***@***.***> wrote: What version of crewai are you using? — Reply to this email directly, view it on GitHub <#897 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAADUQUGNAUFILBSHI6NVNTZLTBTXAVCNFSM6AAAAABKRHRYJWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJZGU4TSNJQGQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[avri-schneider]

try:
pip install crewai[tools] chroma-hnswlib==0.7.5 chromadb==0.5.4

[tang95]

I use a Mac. Before installation, I set HNSWLIB_NO_NATIVE to 1, which resolved the issue. You can try the following:

For Unix-based systems (macOS, Linux):

export HNSWLIB_NO_NATIVE=1

For Windows:

set HNSWLIB_NO_NATIVE=1

Give it a try and see if it works for you!

[Tokeiito]

pip install crewai[tools] chroma-hnswlib==0.7.5 chromadb==0.5.4

This helped on windows installation.