ubuntu install vllm errors

[wangzhiwei-ai]

Building wheels for collected packages: vllm
Building editable for vllm (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building editable for vllm (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [212 lines of output]
running editable_wheel
creating /tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info
writing /tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info/dependency_links.txt
writing requirements to /tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info/requires.txt
writing top-level names to /tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info/top_level.txt
writing manifest file '/tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info/SOURCES.txt'
reading manifest file '/tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file '/tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info/SOURCES.txt'
creating '/tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm-0.1.2.dist-info'
creating /tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm-0.1.2.dist-info/WHEEL
running build_py
running build_ext
/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py:388: UserWarning: The detected CUDA version (11.5) has a minor version mismatch with the version that was used to compile PyTorch (11.7). Most likely this shouldn't be a problem.
warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
building 'vllm.cache_ops' extension
creating /tmp/tmpj94qw548.build-temp/csrc
Emitting ninja build file /tmp/tmpj94qw548.build-temp/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/2] c++ -MMD -MF /tmp/tmpj94qw548.build-temp/csrc/cache.o.d -pthread -B /home/zhiwei.wang/.conda/envs/vllm/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /home/zhiwei.wang/.conda/envs/vllm/include -I/home/zhiwei.wang/.conda/envs/vllm/include -fPIC -O2 -isystem /home/zhiwei.wang/.conda/envs/vllm/include -fPIC -I/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/include -I/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/TH -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/THC -I/home/zhiwei.wang/.conda/envs/vllm/include/python3.9 -c -c /home/zhiwei.wang/serve/vllm/csrc/cache.cpp -o /tmp/tmpj94qw548.build-temp/csrc/cache.o -g -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYBIND11_BUILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=cache_ops -D_GLIBCXX_USE_CXX11_ABI=0
[2/2] /usr/bin/nvcc -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/TH -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/THC -I/home/zhiwei.wang/.conda/envs/vllm/include/python3.9 -c -c /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu -o /tmp/tmpj94qw548.build-temp/csrc/cache_kernels.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_70,code=sm_70 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYBIND11_BUILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=cache_ops -D_GLIBCXX_USE_CXX11_ABI=0
FAILED: /tmp/tmpj94qw548.build-temp/csrc/cache_kernels.o
/usr/bin/nvcc -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/TH -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/THC -I/home/zhiwei.wang/.conda/envs/vllm/include/python3.9 -c -c /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu -o /tmp/tmpj94qw548.build-temp/csrc/cache_kernels.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_70,code=sm_70 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=cache_ops -D_GLIBCXX_USE_CXX11_ABI=0
/home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu(41): warning #1143-D: arithmetic on pointer to void or function type

  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu(42): warning #1143-D: arithmetic on pointer to void or function type

  /tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
            detected during:
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
  (61): here
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
  /tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/c10/core/TensorImpl.h(77): here

  /tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
            detected during:
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  (61): here
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  /tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/ATen/core/qualified_name.h(73): here

  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu(300): warning #550-D: variable "src_key_indices" was set but never used

  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu(301): warning #550-D: variable "src_value_indices" was set but never used

  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu(41): warning #1143-D: arithmetic on pointer to void or function type

  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu(42): warning #1143-D: arithmetic on pointer to void or function type

  /usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
    435 |         function(_Functor&& __f)
        |                                                                                                                                                 ^
  /usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
  /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
    530 |         operator=(_Functor&& __f)
        |                                                                                                                                                  ^
  /usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu: In function ‘void swap_blocks(at::Tensor&, at::Tensor&, const std::map<long int, long int>&)’:
  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu:40:25: warning: pointer of type ‘void *’ used in arithmetic [-Wpointer-arith]
     40 |     cudaMemcpyAsync(
        |                 ~~~~    ^
  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu:40:47: warning: pointer of type ‘void *’ used in arithmetic [-Wpointer-arith]
     40 |     cudaMemcpyAsync(
        |                                               ^
  ninja: build stopped: subcommand failed.
  Traceback (most recent call last):
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
      subprocess.run(
    File "/home/zhiwei.wang/.conda/envs/vllm/lib/python3.9/subprocess.py", line 528, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 155, in run
      self._create_wheel_file(bdist_wheel)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 344, in _create_wheel_file
      files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 267, in _run_build_commands
      self._run_build_subcommands()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 294, in _run_build_subcommands
      self.run_command(name)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 84, in run
      _build_ext.run(self)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
      self.build_extensions()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
      build_ext.build_extensions(self)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
      self._build_extensions_serial()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
      self.build_extension(ext)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
      _build_ext.build_extension(self, ext)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
      objects = self.compiler.compile(
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1574, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  /tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py:988: _DebuggingTips: Problem in editable installation.
  !!

          ********************************************************************************
          An error happened while installing `vllm` in editable mode.

          The following steps are recommended to help debug this problem:

          - Try to install the project normally, without using the editable mode.
            Does the error still persist?
            (If it does, try fixing the problem before attempting the editable mode).
          - If you are using binary extensions, make sure you have all OS-level
            dependencies installed (e.g. compilers, toolchains, binary libraries, ...).
          - Try the latest version of setuptools (maybe the error was already fixed).
          - If you (or your project dependencies) are using any setuptools extension
            or customization, make sure they support the editable mode.

          After following the steps above, if the problem still persists and
          you think this is related to how setuptools handles editable installations,
          please submit a reproducible example
          (see https://stackoverflow.com/help/minimal-reproducible-example) to:

              https://github.com/pypa/setuptools/issues

          See https://setuptools.pypa.io/en/latest/userguide/development_mode.html for details.
          ********************************************************************************

  !!
    cmd_obj.run()
  Traceback (most recent call last):
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
      subprocess.run(
    File "/home/zhiwei.wang/.conda/envs/vllm/lib/python3.9/subprocess.py", line 528, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "/home/zhiwei.wang/.conda/envs/vllm/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/home/zhiwei.wang/.conda/envs/vllm/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/zhiwei.wang/.conda/envs/vllm/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 273, in build_editable
      return hook(wheel_directory, config_settings, metadata_directory)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 445, in build_editable
      return self._build_with_temp_dir(
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
      self.run_setup()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 338, in run_setup
      exec(code, locals())
    File "<string>", line 145, in <module>
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 107, in setup
      return distutils.core.setup(**attrs)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 155, in run
      self._create_wheel_file(bdist_wheel)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 344, in _create_wheel_file
      files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 267, in _run_build_commands
      self._run_build_subcommands()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 294, in _run_build_subcommands
      self.run_command(name)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 84, in run
      _build_ext.run(self)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
      self.build_extensions()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
      build_ext.build_extensions(self)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
      self._build_extensions_serial()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
      self.build_extension(ext)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
      _build_ext.build_extension(self, ext)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
      objects = self.compiler.compile(
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1574, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building editable for vllm
Failed to build vllm
ERROR: Could not build wheels for vllm, which is required to install pyproject.toml-based projects

[yonikremer]

VLLM only supports CUDA 11 to 11.8.
The full system requirements:
OS: Linux
Python: 3.8 or higher
CUDA: 11.0 – 11.8
GPU: compute capability 7.0 or higher (e.g., V100, T4, RTX20xx, A100, L4, etc.)

[dengfenglai321]

Building wheels for collected packages: vllm Building editable for vllm (pyproject.toml) ... error error: subprocess-exited-with-error

× Building editable for vllm (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [212 lines of output] running editable_wheel creating /tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info writing /tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info/PKG-INFO writing dependency_links to /tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info/dependency_links.txt writing requirements to /tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info/requires.txt writing top-level names to /tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info/top_level.txt writing manifest file '/tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info/SOURCES.txt' reading manifest file '/tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' adding license file 'LICENSE' writing manifest file '/tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm.egg-info/SOURCES.txt' creating '/tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm-0.1.2.dist-info' creating /tmp/pip-wheel-xaz048_1/.tmp-wr9v2r2_/vllm-0.1.2.dist-info/WHEEL running build_py running build_ext /tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py:388: UserWarning: The detected CUDA version (11.5) has a minor version mismatch with the version that was used to compile PyTorch (11.7). Most likely this shouldn't be a problem. warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda)) building 'vllm.cache_ops' extension creating /tmp/tmpj94qw548.build-temp/csrc Emitting ninja build file /tmp/tmpj94qw548.build-temp/build.ninja... Compiling objects... Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) [1/2] c++ -MMD -MF /tmp/tmpj94qw548.build-temp/csrc/cache.o.d -pthread -B /home/zhiwei.wang/.conda/envs/vllm/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /home/zhiwei.wang/.conda/envs/vllm/include -I/home/zhiwei.wang/.conda/envs/vllm/include -fPIC -O2 -isystem /home/zhiwei.wang/.conda/envs/vllm/include -fPIC -I/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/include -I/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/TH -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/THC -I/home/zhiwei.wang/.conda/envs/vllm/include/python3.9 -c -c /home/zhiwei.wang/serve/vllm/csrc/cache.cpp -o /tmp/tmpj94qw548.build-temp/csrc/cache.o -g -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYBIND11_BUILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=cache_ops -D_GLIBCXX_USE_CXX11_ABI=0 [2/2] /usr/bin/nvcc -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/TH -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/THC -I/home/zhiwei.wang/.conda/envs/vllm/include/python3.9 -c -c /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu -o /tmp/tmpj94qw548.build-temp/csrc/cache_kernels.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_70,code=sm_70 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYBIND11_BUILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=cache_ops -D_GLIBCXX_USE_CXX11_ABI=0 FAILED: /tmp/tmpj94qw548.build-temp/csrc/cache_kernels.o /usr/bin/nvcc -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/TH -I/tmp/pip-build-env-tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/THC -I/home/zhiwei.wang/.conda/envs/vllm/include/python3.9 -c -c /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu -o /tmp/tmpj94qw548.build-temp/csrc/cache_kernels.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_70,code=sm_70 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=cache_ops -D_GLIBCXX_USE_CXX11_ABI=0 /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu(41): warning #1143-D: arithmetic on pointer to void or function type

  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu(42): warning #1143-D: arithmetic on pointer to void or function type

  /tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
            detected during:
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
  (61): here
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
  /tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/c10/core/TensorImpl.h(77): here

  /tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
            detected during:
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  (61): here
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  /tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/include/ATen/core/qualified_name.h(73): here

  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu(300): warning #550-D: variable "src_key_indices" was set but never used

  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu(301): warning #550-D: variable "src_value_indices" was set but never used

  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu(41): warning #1143-D: arithmetic on pointer to void or function type

  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu(42): warning #1143-D: arithmetic on pointer to void or function type

  /usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
    435 |         function(_Functor&& __f)
        |                                                                                                                                                 ^
  /usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
  /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
    530 |         operator=(_Functor&& __f)
        |                                                                                                                                                  ^
  /usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu: In function ‘void swap_blocks(at::Tensor&, at::Tensor&, const std::map<long int, long int>&)’:
  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu:40:25: warning: pointer of type ‘void *’ used in arithmetic [-Wpointer-arith]
     40 |     cudaMemcpyAsync(
        |                 ~~~~    ^
  /home/zhiwei.wang/serve/vllm/csrc/cache_kernels.cu:40:47: warning: pointer of type ‘void *’ used in arithmetic [-Wpointer-arith]
     40 |     cudaMemcpyAsync(
        |                                               ^
  ninja: build stopped: subcommand failed.
  Traceback (most recent call last):
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
      subprocess.run(
    File "/home/zhiwei.wang/.conda/envs/vllm/lib/python3.9/subprocess.py", line 528, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 155, in run
      self._create_wheel_file(bdist_wheel)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 344, in _create_wheel_file
      files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 267, in _run_build_commands
      self._run_build_subcommands()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 294, in _run_build_subcommands
      self.run_command(name)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 84, in run
      _build_ext.run(self)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
      self.build_extensions()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
      build_ext.build_extensions(self)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
      self._build_extensions_serial()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
      self.build_extension(ext)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
      _build_ext.build_extension(self, ext)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
      objects = self.compiler.compile(
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1574, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  /tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py:988: _DebuggingTips: Problem in editable installation.
  !!

          ********************************************************************************
          An error happened while installing `vllm` in editable mode.

          The following steps are recommended to help debug this problem:

          - Try to install the project normally, without using the editable mode.
            Does the error still persist?
            (If it does, try fixing the problem before attempting the editable mode).
          - If you are using binary extensions, make sure you have all OS-level
            dependencies installed (e.g. compilers, toolchains, binary libraries, ...).
          - Try the latest version of setuptools (maybe the error was already fixed).
          - If you (or your project dependencies) are using any setuptools extension
            or customization, make sure they support the editable mode.

          After following the steps above, if the problem still persists and
          you think this is related to how setuptools handles editable installations,
          please submit a reproducible example
          (see https://stackoverflow.com/help/minimal-reproducible-example) to:

              https://github.com/pypa/setuptools/issues

          See https://setuptools.pypa.io/en/latest/userguide/development_mode.html for details.
          ********************************************************************************

  !!
    cmd_obj.run()
  Traceback (most recent call last):
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
      subprocess.run(
    File "/home/zhiwei.wang/.conda/envs/vllm/lib/python3.9/subprocess.py", line 528, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "/home/zhiwei.wang/.conda/envs/vllm/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/home/zhiwei.wang/.conda/envs/vllm/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/zhiwei.wang/.conda/envs/vllm/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 273, in build_editable
      return hook(wheel_directory, config_settings, metadata_directory)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 445, in build_editable
      return self._build_with_temp_dir(
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
      self.run_setup()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 338, in run_setup
      exec(code, locals())
    File "<string>", line 145, in <module>
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 107, in setup
      return distutils.core.setup(**attrs)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 155, in run
      self._create_wheel_file(bdist_wheel)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 344, in _create_wheel_file
      files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 267, in _run_build_commands
      self._run_build_subcommands()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 294, in _run_build_subcommands
      self.run_command(name)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 84, in run
      _build_ext.run(self)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
      self.build_extensions()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
      build_ext.build_extensions(self)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
      self._build_extensions_serial()
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
      self.build_extension(ext)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
      _build_ext.build_extension(self, ext)
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
      objects = self.compiler.compile(
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1574, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "/tmp/pip-build-env-_tt3rfg4/overlay/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building editable for vllm Failed to build vllm ERROR: Could not build wheels for vllm, which is required to install pyproject.toml-based projects

I have same issue.
my environment is
cuda 11.2
python 3.10.6
v100
linux
.

Did you solve the problem?
could you help me?

[WoosukKwon]

@wangzhiwei-ai Could you let us know your GCC version? It seems it's because you are using a very recent version of GCC. @yumulinfeng1 Could you also provide your GCC version? Sorry for the inconvenience.

[dengfenglai321]

@wangzhiwei-ai Could you let us know your GCC version? It seems it's because you are using a very recent version of GCC. @yumulinfeng1 Could you also provide your GCC version? Sorry for the inconvenience.

gcc (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

so which gcc version is OK?

[wangzhiwei-ai]

hi, GCC version is following: gcc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0 Copyright (C) 2021 Free Software Foundation, Inc. This is free software; see the source for copying conditions.  There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.   Nulooper ***@***.***  

…

------------------ 原始邮件 ------------------ 发件人: ***@***.***>; 发送时间: 2023年7月14日(星期五) 上午10:36 收件人: ***@***.***>; 抄送: ***@***.***>; ***@***.***>; 主题: Re: [vllm-project/vllm] ubuntu install vllm errors (Issue #437) @wangzhiwei-ai Could you let us know your GCC version? It seems it's because you are using a very recent version of GCC. @yumulinfeng1 Could you also provide your GCC version? Sorry for the inconvenience. gcc (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609 Copyright (C) 2015 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. so which gcc version is OK? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[WaterKnight1998]

Same error here, gcc version 11.3.0 @WoosukKwon . Did you solve this in the end? @wangzhiwei-ai

[wangzhiwei-ai]

No. The errors are still there. I have not found solutions.

[iibw]

I solved a similar error with NVIDIA/apex#1491 (comment)

[Colapls]

I encountered the same error under Windows. But I have not found solutions.

[panpan0000]

same here~

[dtnghla]

Please help with this error

[zhangjinnan]

the same