Skip to content

Build Failure When Enabling KleidiAI on ARMv9 in llama-cpp-python ≥ 3.10.0 #2095

@ZIFENG278

Description

@ZIFENG278

When I try to install the latest version of llama-cpp-python and enable KleidiAI on an ARMv9 CPU, I use the following command:

CMAKE_ARGS="-DGGML_NATIVE=OFF -DGGML_CPU_ARM_ARCH=armv9-a+i8mm+dotprod -DGGML_CPU_KLEIDIAI=ON" pip3 install llama-cpp-python

However, during the wheel-building process, it fails with the following error:

      [93/94] : && /usr/bin/cmake -E rm -f vendor/llama.cpp/common/libcommon.a && /usr/bin/aarch64-linux-gnu-ar qc vendor/llama.cpp/common/libcommon.a  vendor/llama.cpp/common/CMakeFiles/build_info.dir/build-info.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/arg.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/chat-parser.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/chat.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/common.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/console.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/json-partial.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/json-schema-to-grammar.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/llguidance.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/log.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/ngram-cache.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/regex-partial.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/sampling.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/speculative.cpp.o && /usr/bin/aarch64-linux-gnu-ranlib vendor/llama.cpp/common/libcommon.a && :
      [94/94] : && /usr/bin/aarch64-linux-gnu-g++ -O3 -DNDEBUG  vendor/llama.cpp/tools/mtmd/CMakeFiles/llama-mtmd-cli.dir/mtmd-cli.cpp.o -o vendor/llama.cpp/tools/mtmd/llama-mtmd-cli  -Wl,-rpath,/tmp/tmpuri70waa/build/vendor/llama.cpp/tools/mtmd:/tmp/tmpuri70waa/build/bin:  vendor/llama.cpp/common/libcommon.a  vendor/llama.cpp/tools/mtmd/libmtmd.so  bin/libllama.so  bin/libggml.so  bin/libggml-cpu.so  bin/libggml-base.so && :
      *** Installing project into wheel...
      -- Install configuration: "Release"
      -- Installing: /tmp/tmpuri70waa/wheel/platlib/lib/libggml-cpu.so
      CMake Error at /tmp/tmpuri70waa/build/_deps/kleidiai_download-build/cmake_install.cmake:52 (file):
        file INSTALL cannot find "/tmp/tmpuri70waa/build/bin/libkleidiai.so": No
        such file or directory.
      Call Stack (most recent call first):
        /tmp/tmpuri70waa/build/vendor/llama.cpp/ggml/src/cmake_install.cmake:65 (include)
        /tmp/tmpuri70waa/build/vendor/llama.cpp/ggml/cmake_install.cmake:47 (include)
        /tmp/tmpuri70waa/build/vendor/llama.cpp/cmake_install.cmake:47 (include)
        /tmp/tmpuri70waa/build/cmake_install.cmake:47 (include)
      
      
      
      *** CMake install failed
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> llama-cpp-python

These CMake flags (-DGGML_NATIVE=OFF -DGGML_CPU_ARM_ARCH=armv9-a+i8mm+dotprod -DGGML_CPU_KLEIDIAI=ON) have already been tested on llama.cpp, and the build succeeds there.

The installation works for llama-cpp-python==3.9.0, but starting from 3.10.0, the build fails with the error above.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions