Trying to compile the latest trtllm (under the v0.12 main branch) in triton 24.07-trtllm-python-py3 reports an error

[gzy19990617]

System Info

triton images:24.07-trtllm-python-py3

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

The image comes with v0.11 trtllm, but v0.11 does not support QWEN2-57b, so I want to compile trtllm to the latest main branch, but it cannot be compiled successfully.

Expected behavior

get trtllm whl

actual behavior

[ 80%] Built target context_attention_src
[ 98%] Built target decoder_attention_src
[ 98%] Built target check_symbol_executor
[ 98%] Built target check_symbol
[ 98%] Built target tensorrt_llm
[100%] Built target nvinfer_plugin_tensorrt_llm
[100%] Built target executorWorker
Requirement already satisfied: pybind11-stubgen in /usr/local/lib/python3.10/dist-packages (2.5.1)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.10/dist-packages/pybind11_stubgen/main.py", line 4, in
main()
File "/usr/local/lib/python3.10/dist-packages/pybind11_stubgen/init.py", line 319, in main
run(
File "/usr/local/lib/python3.10/dist-packages/pybind11_stubgen/init.py", line 358, in run
QualifiedName.from_str(module_name), importlib.import_module(module_name)
File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 674, in _load_unlocked
File "", line 571, in module_from_spec
File "", line 1176, in create_module
File "", line 241, in _call_with_frames_removed
ImportError: /gaoziyuan/gzy/tensorrtllm_backend/tensorrt_llm/tensorrt_llm/bindings.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN12tensorrt_llm8executor14ExecutorConfig25setLogitsPostProcessorMapERKSt13unordered_mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt8functionIFvmRNS0_6TensorERKSt6vectorISC_IiSaIiEESaISE_EERKSt10shared_ptrINS_7runtime10CudaStreamEESt8optionalImEEESt4hashIS8_ESt8equal_toIS8_ESaISt4pairIKS8_SS_EEE
Failed to build pybind11 stubgen: Command '"/usr/bin/python3" -m pybind11_stubgen -o . bindings' returned non-zero exit status 1.

• Building wheel...
  Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in
  main()
  File "/usr/local/lib/python3.10/dist-packages/pyproject_hooks/_in_process/_in_process.py", line 357, in main
  json_out["return_val"] = hook(**hook_input["kwargs"])
  File "/usr/local/lib/python3.10/dist-packages/pyproject_hooks/_in_process/_in_process.py", line 271, in build_wheel
  return _build_backend().build_wheel(
  File "/usr/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 415, in build_wheel
  return self._build_with_temp_dir(
  File "/usr/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
  self.run_setup()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 497, in run_setup
  super().run_setup(setup_script=setup_script)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 313, in run_setup
  exec(code, locals())
  File "", line 84, in
  File "", line 48, in sanity_check
  ImportError: The bindings module does not exist. Please check the package integrity. If you are attempting to use the pip development mode (editable installation), please execute build_wheels.py first, and then run pip install -e ..

ERROR Backend subprocess exited when trying to invoke build_wheel
Traceback (most recent call last):
File "/gaoziyuan/gzy/tensorrtllm_backend/tensorrt_llm/./scripts/build_wheel.py", line 391, in
main(**vars(args))
File "/gaoziyuan/gzy/tensorrtllm_backend/tensorrt_llm/./scripts/build_wheel.py", line 304, in main
build_run(
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '"/usr/bin/python3" -m build /gaoziyuan/gzy/tensorrtllm_backend/tensorrt_llm --skip-dependency-check --no-isolation --wheel --outdir "/gaoziyuan/gzy/tensorrtllm_backend/tensorrt_llm/build"' returned non-zero exit status 1.

additional notes

who can help me.thanks!