Skip to content

Conversation

@lvhan028
Copy link
Collaborator

@lvhan028 lvhan028 commented Nov 7, 2025

No description provided.

@lvhan028 lvhan028 requested a review from CUHKSZzxy November 7, 2025 08:55
@lvhan028 lvhan028 merged commit 28de9a8 into InternLM:main Nov 7, 2025
5 checks passed
@irexyc
Copy link
Collaborator

irexyc commented Nov 10, 2025

2025-11-10 02:29:12,080 - lmdeploy - INFO - async_engine.py:337 - updated backend_config=TurbomindEngineConfig(dtype='auto', model_format=None, tp=8, dp=1, cp=1, device_num=8, attn_tp_size=8, attn_cp_size=1, attn_dp_size=1, mlp_tp_size=8, mlp_dp_size=1, outer_dp_size=1, session_len=40960, max_batch_size=1024, cache_max_entry_count=0.8, cache_chunk_size=-1, cache_block_seq_len=64, enable_prefix_caching=False, quant_policy=0, rope_scaling_factor=0.0, use_logn_attn=False, download_dir=None, revision=None, max_prefill_token_num=8192, num_tokens_per_iter=8192, max_prefill_iters=5, devices=[0, 1, 2, 3, 4, 5, 6, 7], empty_init=False, communicator='nccl', hf_overrides=None, enable_metrics=True)
2025-11-10 02:29:12,312 - lmdeploy - INFO - async_engine.py:400 - enable metrics, with dp: 1 dp_rank: 0
Traceback (most recent call last):
  File "/nvme1/chenxin/ws/lmdeploy/lmdeploy/metrics/loggers.py", line 84, in __init__
    import prometheus_client
ModuleNotFoundError: No module named 'prometheus_client'

@lvhan028
Copy link
Collaborator Author

prometheus_client is already included in runtime_cuda.txt. Please try install it.

2025-11-10 02:29:12,080 - lmdeploy - INFO - async_engine.py:337 - updated backend_config=TurbomindEngineConfig(dtype='auto', model_format=None, tp=8, dp=1, cp=1, device_num=8, attn_tp_size=8, attn_cp_size=1, attn_dp_size=1, mlp_tp_size=8, mlp_dp_size=1, outer_dp_size=1, session_len=40960, max_batch_size=1024, cache_max_entry_count=0.8, cache_chunk_size=-1, cache_block_seq_len=64, enable_prefix_caching=False, quant_policy=0, rope_scaling_factor=0.0, use_logn_attn=False, download_dir=None, revision=None, max_prefill_token_num=8192, num_tokens_per_iter=8192, max_prefill_iters=5, devices=[0, 1, 2, 3, 4, 5, 6, 7], empty_init=False, communicator='nccl', hf_overrides=None, enable_metrics=True)
2025-11-10 02:29:12,312 - lmdeploy - INFO - async_engine.py:400 - enable metrics, with dp: 1 dp_rank: 0
Traceback (most recent call last):
  File "/nvme1/chenxin/ws/lmdeploy/lmdeploy/metrics/loggers.py", line 84, in __init__
    import prometheus_client
ModuleNotFoundError: No module named 'prometheus_client'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants