Not able to build on debian11 arm Python 3.10 #39

Kolumbs · 2022-09-30T06:55:32Z

Cloned this repo and tried following:
pip3 install .

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing /home/juris/Levenshtein
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting rapidfuzz<3.0.0,>=2.3.0
  Using cached https://www.piwheels.org/simple/rapidfuzz/rapidfuzz-2.10.2-py3-none-any.whl (49 kB)
Collecting jarowinkler<2.0.0,>=1.2.2
  Using cached https://www.piwheels.org/simple/jarowinkler/jarowinkler-1.2.3-py3-none-any.whl (6.7 kB)
Building wheels for collected packages: Levenshtein
  Building wheel for Levenshtein (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for Levenshtein (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [92 lines of output]
      Not searching for unused variables given on the command line.
      CMake Error: CMake was unable to find a build program corresponding to "Ninja".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
      -- Configuring incomplete, errors occurred!
      See also "/home/juris/Levenshtein/_cmake_test_compile/build/CMakeFiles/CMakeOutput.log".
      Not searching for unused variables given on the command line.
      -- The C compiler identification is GNU 10.2.1
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- The CXX compiler identification is GNU 10.2.1
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Configuring done
      -- Generating done
      -- Build files have been written to: /home/juris/Levenshtein/_cmake_test_compile/build
      CMake Error at /usr/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
        Could NOT find Python (missing: Interpreter Development.Module)
      Call Stack (most recent call first):
        /usr/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
        /usr/share/cmake-3.18/Modules/FindPython.cmake:436 (find_package_handle_standard_args)
        CMakeLists.txt:25 (find_package)
      
      
      -- Configuring incomplete, errors occurred!
      See also "/home/juris/Levenshtein/CMakeFiles/CMakeOutput.log".
        File "/tmp/pip-build-env-np1mmvek/overlay/lib/python3.10/site-packages/skbuild/setuptools_wrap.py", line 637, in setup
          env = cmkr.configure(
        File "/tmp/pip-build-env-np1mmvek/overlay/lib/python3.10/site-packages/skbuild/cmaker.py", line 328, in configure
          raise SKBuildError(
      
      
      --------------------------------------------------------------------------------
      -- Trying "Ninja" generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying "Ninja" generator - failure
      --------------------------------------------------------------------------------
      
      
      
      --------------------------------------------------------------------------------
      -- Trying "Unix Makefiles" generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying "Unix Makefiles" generator - success
      --------------------------------------------------------------------------------
      
      Configuring Project
        Working directory:
          /home/juris/Levenshtein/_skbuild/linux-armv7l-3.10/cmake-build
        Command:
          cmake /home/juris/Levenshtein -G 'Unix Makefiles' -DCMAKE_INSTALL_PREFIX:PATH=/home/juris/Levenshtein/_skbuild/linux-armv7l-3.10/cmake-install -DPYTHON_VERSION_STRING:STRING=3.10.7 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-np1mmvek/overlay/lib/python3.10/site-packages/skbuild/resources/cmake -DPython3_EXECUTABLE:FILEPATH=/usr/local/bin/python3.10 -DPython3_INCLUDE_DIR:PATH=/usr/local/include/python3.10 -DPython3_LIBRARY:PATH=/usr/local/lib/libpython3.10.a -DPython_EXECUTABLE:FILEPATH=/usr/local/bin/python3.10 -DPython_INCLUDE_DIR:PATH=/usr/local/include/python3.10 -DPython_LIBRARY:PATH=/usr/local/lib/libpython3.10.a -DPYTHON_EXECUTABLE:FILEPATH=/usr/local/bin/python3.10 -DPYTHON_INCLUDE_DIR:PATH=/usr/local/include/python3.10 -DPYTHON_LIBRARY:PATH=/usr/local/lib/libpython3.10.a -DCMAKE_BUILD_TYPE:STRING=Release
      
      Traceback (most recent call last):
      
      An error occurred while configuring with CMake.
        Command:
          cmake /home/juris/Levenshtein -G 'Unix Makefiles' -DCMAKE_INSTALL_PREFIX:PATH=/home/juris/Levenshtein/_skbuild/linux-armv7l-3.10/cmake-install -DPYTHON_VERSION_STRING:STRING=3.10.7 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-np1mmvek/overlay/lib/python3.10/site-packages/skbuild/resources/cmake -DPython3_EXECUTABLE:FILEPATH=/usr/local/bin/python3.10 -DPython3_INCLUDE_DIR:PATH=/usr/local/include/python3.10 -DPython3_LIBRARY:PATH=/usr/local/lib/libpython3.10.a -DPython_EXECUTABLE:FILEPATH=/usr/local/bin/python3.10 -DPython_INCLUDE_DIR:PATH=/usr/local/include/python3.10 -DPython_LIBRARY:PATH=/usr/local/lib/libpython3.10.a -DPYTHON_EXECUTABLE:FILEPATH=/usr/local/bin/python3.10 -DPYTHON_INCLUDE_DIR:PATH=/usr/local/include/python3.10 -DPYTHON_LIBRARY:PATH=/usr/local/lib/libpython3.10.a -DCMAKE_BUILD_TYPE:STRING=Release
        Source directory:
          /home/juris/Levenshtein
        Working directory:
          /home/juris/Levenshtein/_skbuild/linux-armv7l-3.10/cmake-build
      Please see CMake's output for more information.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for Levenshtein
Failed to build Levenshtein
ERROR: Could not build wheels for Levenshtein, which is required to install pyproject.toml-based projects

I am not an user of cmake, but from this I understand cmake could not find header files for Python. However I have Python built and installed from source. Additionally plain cmake would find the Python with following simplification of your CMakeLists.txt:

cmake_minimum_required(VERSION 3.12...3.24)

set(Python_FIND_IMPLEMENTATIONS CPython PyPy)

project(Levenshtein LANGUAGES C CXX)

if(CMAKE_VERSION VERSION_LESS 3.18)
    find_package(Python COMPONENTS Interpreter Development REQUIRED)
else()
    set(Python_ARTIFACTS_INTERACTIVE TRUE)
    find_package(Python COMPONENTS Interpreter Development.Module REQUIRED)
endif()

execute_process(
    COMMAND "${Python_EXECUTABLE}" -c
            "import sysconfig; print(sysconfig.get_config_var('EXT_SUFFIX').split('.')[1])"
    OUTPUT_VARIABLE Python_SOABI
)
message(STATUS "Python was found and returned: ${Python_SOABI}")

When I run on the above file cmake .

-- The C compiler identification is GNU 10.2.1
-- The CXX compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Python was found and returned: cpython-310-arm-linux-gnueabihf

-- Configuring done
-- Generating done
-- Build files have been written to: /home/juris/cmake_test

lsb_release -a

Description:	Raspbian GNU/Linux 11 (bullseye)
Release:	11
Codename:	bullseye

uname -a
Linux 5.10.92-v7l+ #1514 SMP Mon Jan 17 17:38:03 GMT 2022 armv7l GNU/Linux

The text was updated successfully, but these errors were encountered:

maxbachmann · 2022-09-30T10:08:41Z

Ah yes that is a bug in scikit-build: scikit-build/scikit-build#718, so you can workaround this by hardcoding the version in the pyproject.toml: "scikit-build==0.14.1". I am experiencing this when developing on Fedora as well.

I still need to test whether the build passes with 0.14.1 everywhere. If it does I I will hardcode it until there is a fix in scikit-build.

maxbachmann · 2022-09-30T10:15:27Z

Uh this requires more work, since the piwheels build fails.
Seems like either their compiler does not support C++17 or there is a compiler bug I need to workaround ...

Kolumbs · 2022-09-30T11:03:23Z

well, I don't observe anymore fails after forcing scikit-build to use 0.14.1 version:

$ pip3 install .
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing /home/juris/Levenshtein
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: rapidfuzz<3.0.0,>=2.3.0 in /home/juris/.local/lib/python3.10/site-packages (from Levenshtein==0.20.5) (2.10.2)
Requirement already satisfied: jarowinkler<2.0.0,>=1.2.2 in /home/juris/.local/lib/python3.10/site-packages (from rapidfuzz<3.0.0,>=2.3.0->Levenshtein==0.20.5) (1.2.3)
Building wheels for collected packages: Levenshtein
  Building wheel for Levenshtein (pyproject.toml) ... done
  Created wheel for Levenshtein: filename=Levenshtein-0.20.5-cp310-cp310-linux_armv7l.whl size=165564 sha256=5b6eb87aed1223ee3b78db4ffff1f6c334b37a888b426aadf822da60b3b0b129
  Stored in directory: /tmp/pip-ephem-wheel-cache-m9xx6z2k/wheels/b3/27/31/93160a7a7432fa29e69bfdcd8d0b87fe1c99a6a6b5950d1ab5
Successfully built Levenshtein
Installing collected packages: Levenshtein
  Attempting uninstall: Levenshtein
    Found existing installation: Levenshtein 0.20.5
    Uninstalling Levenshtein-0.20.5:
      Successfully uninstalled Levenshtein-0.20.5
Successfully installed Levenshtein-0.20.5

maxbachmann · 2022-09-30T11:06:05Z

You should probably build rapidfuzz from source as well, since the build on piwheels is broken right now due to this bug in scikit-build + a bug in gcc ...
Otherwise you only have the pure Python version, which is quite a bit slower.

Kolumbs · 2022-09-30T11:30:00Z

I think I got from pip some prebuilt rapidfuzz, so I have everything allright now:

Python 3.10.7 (main, Sep 25 2022, 13:32:17) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from rapidfuzz import process, fuzz
>>> choices = ["Atlanta Falcons", "New York Jets", "New York Giants", "Dallas Cowboys"]
>>> process.extract("new york jets", choices, scorer=fuzz.WRatio, limit=2)
[('New York Jets', 100.0, 1), ('New York Giants', 78.57142857142857, 2)]

maxbachmann · 2022-09-30T11:30:49Z

Can you try:

from rapidfuzz import fuzz_cpp

Kolumbs · 2022-09-30T11:32:45Z

ah well this then means I am using the slow version:

Python 3.10.7 (main, Sep 25 2022, 13:32:17) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from rapidfuzz import fuzz_cpp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'fuzz_cpp' from 'rapidfuzz'

maxbachmann · 2022-09-30T11:34:22Z

Yes I expected this, since the arm build on https://www.piwheels.org/project/rapidfuzz/ was silently broken. I hope this will be fixed by the changes in rapidfuzz/RapidFuzz@24bf798

Kolumbs · 2022-09-30T11:50:22Z

i took your latest commit from rapidfuzz and I am afraid the problem is still there. This is what I got after build in rapidfuzz repo:

WARNING: The C extension could not be compiled, speedups are not enabled.
Plain-Python build succeeded.

maxbachmann · 2022-09-30T11:55:37Z

Could you try:

RAPIDFUZZ_BUILD_EXTENSION="1" pip install . -v

to check why the c++ build fails?

Kolumbs · 2022-09-30T12:11:10Z

this would be the full result: dump.txt

maxbachmann · 2022-09-30T12:13:41Z

It appears you did not clone the submodules. You should be able to get them using:

git submodule update --init

Kolumbs · 2022-09-30T12:32:30Z

yes sorry those submodules were culprit, it built the thing, not sure though why this would still complain:

>>> from rapidfuzz import fuzz_cpp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'fuzz_cpp' from 'rapidfuzz'

maxbachmann · 2022-09-30T12:36:34Z

Hm can you check whether the fuzz_cpp.*.so file is present in the corresponding site-packages directory (site-packages/rapidfuzz/fuzz)?
If it is you can try to cd into the directory and try to import fuzz_cpp in there.

maxbachmann · 2022-09-30T12:39:05Z

if the file is present, what is the file name? (to check whether the platform tag is correct)

Kolumbs · 2022-10-01T04:26:49Z

I have reinstalled again and it is ok:

Python 3.10.7 (main, Sep 25 2022, 13:32:17) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from rapidfuzz import fuzz_cpp
>>>

sorry for fake claims, apparently at the first reinstallation maybe I took some previous wheel or maybe even accidentally installed rapidfuzz-2.10.3-py3-none-any.whl instead of rapidfuzz-2.10.3-cp310-cp310-linux_armv7l.whl . Not sure. Anyhow all good now. Thanks.
Not sure if I should close this or you want to close it as soon as the scikit-build regression is fixed, which was the original cause of this bug for Levenshtein.

maxbachmann · 2022-10-01T12:03:38Z

I think we should keep this open, for anyone else running into this issue until it is resolved in scikit-build.

maxbachmann · 2022-11-07T15:36:04Z

This should be fixed with the latest version of scikit-build

maxbachmann added the bug Something isn't working label Sep 30, 2022

maxbachmann closed this as completed Nov 7, 2022

ovv mentioned this issue Apr 12, 2023

Fails to build debian11/python 3.9: Could NOT find Python (missing: Interpreter Development.Module) #52

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Not able to build on debian11 arm Python 3.10 #39

Not able to build on debian11 arm Python 3.10 #39

Kolumbs commented Sep 30, 2022 •

edited

Loading

maxbachmann commented Sep 30, 2022 •

edited

Loading

maxbachmann commented Sep 30, 2022

Kolumbs commented Sep 30, 2022

maxbachmann commented Sep 30, 2022

Kolumbs commented Sep 30, 2022

maxbachmann commented Sep 30, 2022

Kolumbs commented Sep 30, 2022 •

edited

Loading

maxbachmann commented Sep 30, 2022

Kolumbs commented Sep 30, 2022

maxbachmann commented Sep 30, 2022

Kolumbs commented Sep 30, 2022

maxbachmann commented Sep 30, 2022

Kolumbs commented Sep 30, 2022

maxbachmann commented Sep 30, 2022

maxbachmann commented Sep 30, 2022 •

edited

Loading

Kolumbs commented Oct 1, 2022

maxbachmann commented Oct 1, 2022

maxbachmann commented Nov 7, 2022

Not able to build on debian11 arm Python 3.10 #39

Not able to build on debian11 arm Python 3.10 #39

Comments

Kolumbs commented Sep 30, 2022 • edited Loading

maxbachmann commented Sep 30, 2022 • edited Loading

maxbachmann commented Sep 30, 2022

Kolumbs commented Sep 30, 2022

maxbachmann commented Sep 30, 2022

Kolumbs commented Sep 30, 2022

maxbachmann commented Sep 30, 2022

Kolumbs commented Sep 30, 2022 • edited Loading

maxbachmann commented Sep 30, 2022

Kolumbs commented Sep 30, 2022

maxbachmann commented Sep 30, 2022

Kolumbs commented Sep 30, 2022

maxbachmann commented Sep 30, 2022

Kolumbs commented Sep 30, 2022

maxbachmann commented Sep 30, 2022

maxbachmann commented Sep 30, 2022 • edited Loading

Kolumbs commented Oct 1, 2022

maxbachmann commented Oct 1, 2022

maxbachmann commented Nov 7, 2022

Kolumbs commented Sep 30, 2022 •

edited

Loading

maxbachmann commented Sep 30, 2022 •

edited

Loading

Kolumbs commented Sep 30, 2022 •

edited

Loading

maxbachmann commented Sep 30, 2022 •

edited

Loading