[Bug]: vllm0.4.0 post1，双卡加载qwen72b报错：NameError: name 'ncclGetVersion' is not defined

[wzjie]

Your current environment

环境Python 3.10.0，torch2.1.2+cu118

🐛 Describe the bug

vllm0.4.0 post1，双卡加载qwen72b报错：NameError: name 'ncclGetVersion' is not defined

[wzjie]

详细错误：
File "vllm_examples/test_vllm.py", line 18, in
llm = LLM(model="model/Qwen1.5-72B-Chat", tensor_parallel_size=2, gpu_memory_utilization=0.95, trust_remote_code=True, enable_lora=True)
File "miniconda3/envs/chain_env/lib/python3.10/site-packages/vllm/entrypoints/llm.py", line 112, in init
self.llm_engine = LLMEngine.from_engine_args(
File "miniconda3/envs/chain_env/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 196, in from_engine_args
engine = cls(
File "miniconda3/envs/chain_env/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 110, in init
self.model_executor = executor_class(model_config, cache_config,
File "miniconda3/envs/chain_env/lib/python3.10/site-packages/vllm/executor/ray_gpu_executor.py", line 62, in init
self._init_workers_ray(placement_group)
File "miniconda3/envs/chain_env/lib/python3.10/site-packages/vllm/executor/ray_gpu_executor.py", line 191, in _init_workers_ray
self._run_workers("init_device")
File "miniconda3/envs/chain_env/lib/python3.10/site-packages/vllm/executor/ray_gpu_executor.py", line 324, in _run_workers
driver_worker_output = getattr(self.driver_worker,
File "miniconda3/envs/chain_env/lib/python3.10/site-packages/vllm/worker/worker.py", line 100, in init_device
init_distributed_environment(self.parallel_config, self.rank,
File "miniconda3/envs/chain_env/lib/python3.10/site-packages/vllm/worker/worker.py", line 287, in init_distributed_environment
pynccl_utils.init_process_group(
File "miniconda3/envs/chain_env/lib/python3.10/site-packages/vllm/model_executor/parallel_utils/pynccl_utils.py", line 45, in init_process_group
logger.info(f"vLLM is using nccl=={ncclGetVersion()}")

[youkaichao]

Before this error message, did you see logging information like Failed to import NCCL library:?

[wzjie]

Before this error message, did you see logging information like Failed to import NCCL library:?

yes, the error is:ERROR 04-08 17:04:49 pynccl.py:53] Failed to load NCCL library from libnccl.so.2 .It is expected if you are not running on NVIDIA/AMD GPUs.Otherwise please set the environment variable VLLM_NCCL_SO_PATH to point to the correct nccl library path.
INFO 04-08 17:04:49 pynccl_utils.py:17] Failed to import NCCL library: libnccl.so.2: cannot open shared object file: No such file or directory
INFO 04-08 17:04:49 pynccl_utils.py:18] It is expected if you are not running on NVIDIA GPUs.
INFO 04-08 17:04:50 selector.py:51] Cannot use FlashAttention because the package is not found. Please install it for better performance.
INFO 04-08 17:04:50 selector.py:25] Using XFormers backend.
�[36m(RayWorkerVllm pid=2915268)�[0m ERROR 04-08 17:04:51 pynccl.py:53] Failed to load NCCL library from libnccl.so.2 .It is expected if you are not running on NVIDIA/AMD GPUs.Otherwise please set the environment variable VLLM_NCCL_SO_PATH to point to the correct nccl library path.
�[36m(RayWorkerVllm pid=2915268)�[0m INFO 04-08 17:04:51 pynccl_utils.py:17] Failed to import NCCL library: libnccl.so.2: cannot open shared object file: No such file or directory
�[36m(RayWorkerVllm pid=2915268)�[0m INFO 04-08 17:04:51 pynccl_utils.py:18] It is expected if you are not running on NVIDIA GPUs.
�[36m(RayWorkerVllm pid=2915268)�[0m INFO 04-08 17:04:52 selector.py:51] Cannot use FlashAttention because the package is not found. Please install it for better performance.
�[36m(RayWorkerVllm pid=2915268)�[0m INFO 04-08 17:04:52 selector.py:25] Using XFormers backend.
Traceback (most recent call last):
File "vllm_examples/test_vllm.py", line 18, in
llm = LLM(model="model/Qwen1.5-72B-Chat", tensor_parallel_size=2, gpu_memory_utilization=0.95, trust_remote_code=True, enable_lora=True)

[youkaichao]

Then it is clear your error comes from failing to load libnccl.so.2 . It should be a problem of your pytorch installation.

By the way, you can try to build from source in the current main, which might solve this issue. After #3805 , vllm will download a proper nccl version for you.