本地启动无法用gpu拉起自定义路径的本地模型 ValueError: [address=0.0.0.0:11083, pid=11576] Worker 0.0.0.0:11083 cannot use the GPUs with these indexes: [2]. Worker 0.0.0.0:11083 can only see these GPUs: [].

[shendepuren]

1. 启动方式 xinference-local --host 0.0.0.0 --port 9997
2. 版本 0.12.3
3. 报错
   2024-07-01 19:52:53,099 xinference.api.restful_api 11474 ERROR [address=0.0.0.0:11083, pid=11576] Worker 0.0.0.0:11083 cannot use the GPUs with these indexes: [2]. Worker 0.0.0.0:11083 can only see these GPUs: []. Traceback (most recent call last): File "/data/miniconda3/envs/xinference/lib/python3.10/site-packages/xinference/api/restful_api.py", line 822, in launch_model model_uid = await (await self._get_supervisor_ref()).launch_builtin_model( File "/data/miniconda3/envs/xinference/lib/python3.10/site-packages/xoscar/backends/context.py", line 227, in send return self._process_result_message(result) File "/data/miniconda3/envs/xinference/lib/python3.10/site-packages/xoscar/backends/context.py", line 102, in _process_result_message raise message.as_instanceof_cause() File "/data/miniconda3/envs/xinference/lib/python3.10/site-packages/xoscar/backends/pool.py", line 659, in send result = await self._run_coro(message.message_id, coro) File "/data/miniconda3/envs/xinference/lib/python3.10/site-packages/xoscar/backends/pool.py", line 370, in _run_coro return await coro File "/data/miniconda3/envs/xinference/lib/python3.10/site-packages/xoscar/api.py", line 384, in __on_receive__ return await super().__on_receive__(message) # type: ignore File "xoscar/core.pyx", line 558, in __on_receive__ raise ex File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.__on_receive__ async with self._lock: File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.__on_receive__ with debug_async_timeout('actor_lock_timeout', File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__ result = await result File "/data/miniconda3/envs/xinference/lib/python3.10/site-packages/xinference/core/supervisor.py", line 871, in launch_builtin_model await _launch_model() File "/data/miniconda3/envs/xinference/lib/python3.10/site-packages/xinference/core/supervisor.py", line 835, in _launch_model await _launch_one_model(rep_model_uid) File "/data/miniconda3/envs/xinference/lib/python3.10/site-packages/xinference/core/supervisor.py", line 816, in _launch_one_model await worker_ref.launch_builtin_model( File "xoscar/core.pyx", line 284, in __pyx_actor_method_wrapper async with lock: File "xoscar/core.pyx", line 287, in xoscar.core.__pyx_actor_method_wrapper result = await result File "/data/miniconda3/envs/xinference/lib/python3.10/site-packages/xinference/core/utils.py", line 45, in wrapped ret = await func(*args, **kwargs) File "/data/miniconda3/envs/xinference/lib/python3.10/site-packages/xinference/core/worker.py", line 643, in launch_builtin_model subpool_address, devices = await self._create_subpool( File "/data/miniconda3/envs/xinference/lib/python3.10/site-packages/xinference/core/worker.py", line 490, in _create_subpool devices = await self.allocate_devices_with_gpu_idx( File "/data/miniconda3/envs/xinference/lib/python3.10/site-packages/xinference/core/worker.py", line 400, in allocate_devices_with_gpu_idx raise ValueError( ValueError: [address=0.0.0.0:11083, pid=11576] Worker 0.0.0.0:11083 cannot use the GPUs with these indexes: [2]. Worker 0.0.0.0:11083 can only see these GPUs: [].

模型配置：
{ "version": 1, "context_length": 2048, "model_name": "chatglm3-6b-base", "model_lang": [ "en", "zh" ], "model_ability": [ "generate", "chat" ], "model_description": "This is a custom model description.", "model_family": "chatglm3", "model_specs": [ { "model_format": "pytorch", "model_size_in_billions": 6, "quantizations": [ "none" ], "model_id": null, "model_hub": "huggingface", "model_uri": "/data/ai_server/hub/chatglm3-6b-base", "model_revision": null } ], "prompt_style": { "style_name": "CHATGLM3", "system_prompt": "", "roles": [ "user", "assistant" ], "intra_message_sep": "", "inter_message_sep": "", "stop": [ "<|user|>", "<|observation|>" ], "stop_token_ids": [ 64795, 64797, 2 ] }, "is_builtin": false }
5. 操作

[ChengjieLi28]

xinference识别不到你的GPU，检查自己的环境。