[Bug]: OpenAI-Compatible Server cannot be requested #15675
Description
Your current environment
The output of `python collect_env.py`
INFO 03-28 11:14:11 [__init__.py:239] Automatically detected platform cuda.
Collecting environment information...
PyTorch version: 2.6.0+cu124
Is debug build: False
CUDA used to build PyTorch: 12.4
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.5 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35
Python version: 3.12.9 | packaged by Anaconda, Inc. | (main, Feb 6 2025, 18:56:27) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 12.1.66
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4060 Laptop GPU
Nvidia driver version: 560.94
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.8.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.8.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.8.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.8.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.8.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.8.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.8.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 39 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 32
On-line CPU(s) list: 0-31
Vendor ID: GenuineIntel
Model name: Intel(R) Core(TM) i9-14900HX
CPU family: 6
Model: 183
Thread(s) per core: 2
Core(s) per socket: 16
Socket(s): 1
Stepping: 1
BogoMIPS: 4838.39
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves avx_vnni umip waitpkg gfni vaes vpclmulqdq rdpid movdiri movdir64b fsrm md_clear serialize flush_l1d arch_capabilities
Virtualization: VT-x
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 768 KiB (16 instances)
L1i cache: 512 KiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 36 MiB (1 instance)
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Vulnerable: No microcode
Vulnerability Retbleed: Mitigation; Enhanced IBRS
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced / Automatic IBRS; IBPB conditional; RSB filling; PBRSB-eIBRS SW sequence; BHI BHI_DIS_S
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] nvidia-cublas-cu12==12.4.5.8
[pip3] nvidia-cuda-cupti-cu12==12.4.127
[pip3] nvidia-cuda-nvrtc-cu12==12.4.127
[pip3] nvidia-cuda-runtime-cu12==12.4.127
[pip3] nvidia-cudnn-cu12==9.1.0.70
[pip3] nvidia-cufft-cu12==11.2.1.3
[pip3] nvidia-curand-cu12==10.3.5.147
[pip3] nvidia-cusolver-cu12==11.6.1.9
[pip3] nvidia-cusparse-cu12==12.3.1.170
[pip3] nvidia-cusparselt-cu12==0.6.2
[pip3] nvidia-nccl-cu12==2.21.5
[pip3] nvidia-nvjitlink-cu12==12.4.127
[pip3] nvidia-nvtx-cu12==12.4.127
[pip3] pyzmq==26.3.0
[pip3] torch==2.6.0
[pip3] torchaudio==2.6.0
[pip3] torchvision==0.21.0
[pip3] transformers==4.50.1
[pip3] triton==3.2.0
[conda] blas 1.0 mkl https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
[conda] cuda-cudart 12.1.105 0 nvidia
[conda] cuda-cupti 12.1.105 0 nvidia
[conda] cuda-libraries 12.1.0 0 nvidia
[conda] cuda-nvrtc 12.1.105 0 nvidia
[conda] cuda-nvtx 12.1.105 0 nvidia
[conda] cuda-opencl 12.8.90 0 nvidia
[conda] cuda-runtime 12.1.0 0 nvidia
[conda] cuda-version 12.8 3 nvidia
[conda] ffmpeg 4.3 hf484d3e_0 pytorch
[conda] libcublas 12.1.0.26 0 nvidia
[conda] libcufft 11.0.2.4 0 nvidia
[conda] libcufile 1.13.1.3 0 nvidia
[conda] libcurand 10.3.9.90 0 nvidia
[conda] libcusolver 11.4.4.55 0 nvidia
[conda] libcusparse 12.0.2.55 0 nvidia
[conda] libjpeg-turbo 2.0.0 h9bf148f_0 pytorch
[conda] libnpp 12.0.2.50 0 nvidia
[conda] libnvjitlink 12.1.105 0 nvidia
[conda] libnvjpeg 12.1.1.14 0 nvidia
[conda] mkl 2023.1.0 h213fc3f_46344 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] mkl-service 2.4.0 py312h5eee18b_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] mkl_fft 1.3.11 py312h5eee18b_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] mkl_random 1.2.8 py312h526ad5a_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] numpy 1.26.4 pypi_0 pypi
[conda] nvidia-cublas-cu12 12.4.5.8 pypi_0 pypi
[conda] nvidia-cuda-cupti-cu12 12.4.127 pypi_0 pypi
[conda] nvidia-cuda-nvrtc-cu12 12.4.127 pypi_0 pypi
[conda] nvidia-cuda-runtime-cu12 12.4.127 pypi_0 pypi
[conda] nvidia-cudnn-cu12 9.1.0.70 pypi_0 pypi
[conda] nvidia-cufft-cu12 11.2.1.3 pypi_0 pypi
[conda] nvidia-curand-cu12 10.3.5.147 pypi_0 pypi
[conda] nvidia-cusolver-cu12 11.6.1.9 pypi_0 pypi
[conda] nvidia-cusparse-cu12 12.3.1.170 pypi_0 pypi
[conda] nvidia-cusparselt-cu12 0.6.2 pypi_0 pypi
[conda] nvidia-nccl-cu12 2.21.5 pypi_0 pypi
[conda] nvidia-nvjitlink-cu12 12.4.127 pypi_0 pypi
[conda] nvidia-nvtx-cu12 12.4.127 pypi_0 pypi
[conda] pytorch-cuda 12.1 ha16c6d3_6 pytorch
[conda] pytorch-mutex 1.0 cuda pytorch
[conda] pyzmq 26.3.0 pypi_0 pypi
[conda] torch 2.6.0 pypi_0 pypi
[conda] torchaudio 2.6.0 pypi_0 pypi
[conda] torchvision 0.21.0 pypi_0 pypi
[conda] transformers 4.50.1 pypi_0 pypi
[conda] triton 3.2.0 pypi_0 pypi
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.8.2
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0 CPU Affinity NUMA Affinity GPU NUMA ID
GPU0 X N/A
Legend:
X = Self
SYS = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
PHB = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
PXB = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
PIX = Connection traversing at most a single PCIe bridge
NV# = Connection traversing a bonded set of # NVLinks
LD_LIBRARY_PATH=/usr/local/cuda/lib64
NCCL_CUMEM_ENABLE=0
TORCHINDUCTOR_COMPILE_THREADS=1
CUDA_MODULE_LOADING=LAZY
🐛 Describe the bug
I use this command start vllm server
python -m vllm.entrypoints.openai.api_server --model /mnt/f/wsl/ubuntu2204/models/Qwen2.5-0.5B --gpu_memory_utilization=0.99 --max-model-len=2048
and output the following information
INFO 03-28 11:37:50 [__init__.py:239] Automatically detected platform cuda.
INFO 03-28 11:37:52 [api_server.py:981] vLLM API server version 0.8.2
INFO 03-28 11:37:52 [api_server.py:982] args: Namespace(host=None, port=8000, uvicorn_log_level='info', disable_uvicorn_access_log=False, allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api_key=None, lora_modules=None, prompt_adapters=None, chat_template=None, chat_template_content_format='auto', response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None, enable_ssl_refresh=False, ssl_cert_reqs=0, root_path=None, middleware=[], return_tokens_as_token_ids=False, disable_frontend_multiprocessing=False, enable_request_id_headers=False, enable_auto_tool_choice=False, tool_call_parser=None, tool_parser_plugin='', model='/mnt/f/wsl/ubuntu2204/models/Qwen2.5-0.5B', task='auto', tokenizer=None, hf_config_path=None, skip_tokenizer_init=False, revision=None, code_revision=None, tokenizer_revision=None, tokenizer_mode='auto', trust_remote_code=False, allowed_local_media_path=None, download_dir=None, load_format='auto', config_format=<ConfigFormat.AUTO: 'auto'>, dtype='auto', kv_cache_dtype='auto', max_model_len=2048, guided_decoding_backend='xgrammar', logits_processor_pattern=None, model_impl='auto', distributed_executor_backend=None, pipeline_parallel_size=1, tensor_parallel_size=1, enable_expert_parallel=False, max_parallel_loading_workers=None, ray_workers_use_nsight=False, block_size=None, enable_prefix_caching=None, disable_sliding_window=False, use_v2_block_manager=True, num_lookahead_slots=0, seed=None, swap_space=4, cpu_offload_gb=0, gpu_memory_utilization=0.99, num_gpu_blocks_override=None, max_num_batched_tokens=None, max_num_partial_prefills=1, max_long_partial_prefills=1, long_prefill_token_threshold=0, max_num_seqs=None, max_logprobs=20, disable_log_stats=False, quantization=None, rope_scaling=None, rope_theta=None, hf_overrides=None, enforce_eager=False, max_seq_len_to_capture=8192, disable_custom_all_reduce=False, tokenizer_pool_size=0, tokenizer_pool_type='ray', tokenizer_pool_extra_config=None, limit_mm_per_prompt=None, mm_processor_kwargs=None, disable_mm_preprocessor_cache=False, enable_lora=False, enable_lora_bias=False, max_loras=1, max_lora_rank=16, lora_extra_vocab_size=256, lora_dtype='auto', long_lora_scaling_factors=None, max_cpu_loras=None, fully_sharded_loras=False, enable_prompt_adapter=False, max_prompt_adapters=1, max_prompt_adapter_token=0, device='auto', num_scheduler_steps=1, use_tqdm_on_load=True, multi_step_stream_outputs=True, scheduler_delay_factor=0.0, enable_chunked_prefill=None, speculative_config=None, speculative_model=None, speculative_model_quantization=None, num_speculative_tokens=None, speculative_disable_mqa_scorer=False, speculative_draft_tensor_parallel_size=None, speculative_max_model_len=None, speculative_disable_by_batch_size=None, ngram_prompt_lookup_max=None, ngram_prompt_lookup_min=None, spec_decoding_acceptance_method='rejection_sampler', typical_acceptance_sampler_posterior_threshold=None, typical_acceptance_sampler_posterior_alpha=None, disable_logprobs_during_spec_decoding=None, model_loader_extra_config=None, ignore_patterns=[], preemption_mode=None, served_model_name=None, qlora_adapter_name_or_path=None, show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, disable_async_output_proc=False, scheduling_policy='fcfs', scheduler_cls='vllm.core.scheduler.Scheduler', override_neuron_config=None, override_pooler_config=None, compilation_config=None, kv_transfer_config=None, worker_cls='auto', worker_extension_cls='', generation_config='auto', override_generation_config=None, enable_sleep_mode=False, calculate_kv_scales=False, additional_config=None, enable_reasoning=False, reasoning_parser=None, disable_cascade_attn=False, disable_log_requests=False, max_log_len=None, disable_fastapi_docs=False, enable_prompt_tokens_details=False, enable_server_load_tracking=False)
INFO 03-28 11:37:56 [config.py:585] This model supports multiple tasks: {'classify', 'reward', 'generate', 'embed', 'score'}. Defaulting to 'generate'.
INFO 03-28 11:37:56 [config.py:1697] Chunked prefill is enabled with max_num_batched_tokens=2048.
INFO 03-28 11:37:57 [core.py:54] Initializing a V1 LLM engine (v0.8.2) with config: model='/mnt/f/wsl/ubuntu2204/models/Qwen2.5-0.5B', speculative_config=None, tokenizer='/mnt/f/wsl/ubuntu2204/models/Qwen2.5-0.5B', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=2048, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='xgrammar', reasoning_backend=None), observability_config=ObservabilityConfig(show_hidden_metrics=False, otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=None, served_model_name=/mnt/f/wsl/ubuntu2204/models/Qwen2.5-0.5B, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=True, disable_mm_preprocessor_cache=False, mm_processor_kwargs=None, pooler_config=None, compilation_config={"level":3,"custom_ops":["none"],"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output"],"use_inductor":true,"compile_sizes":[],"use_cudagraph":true,"cudagraph_num_of_warmups":1,"cudagraph_capture_sizes":[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"max_capture_size":512}
WARNING 03-28 11:37:58 [utils.py:2321] Methods determine_num_available_blocks,device_config,get_cache_block_size_bytes,initialize_cache not implemented in <vllm.v1.worker.gpu_worker.Worker object at 0x7ff0f054e390>
[W328 11:38:09.607568369 socket.cpp:204] [c10d] The hostname of the client socket cannot be retrieved. err=-3
[W328 11:38:19.619372363 socket.cpp:204] [c10d] The hostname of the client socket cannot be retrieved. err=-3
INFO 03-28 11:38:20 [parallel_state.py:954] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0
WARNING 03-28 11:38:20 [interface.py:303] Using 'pin_memory=False' as WSL is detected. This may slow down the performance.
INFO 03-28 11:38:20 [cuda.py:220] Using Flash Attention backend on V1 engine.
INFO 03-28 11:38:20 [gpu_model_runner.py:1174] Starting to load model /mnt/f/wsl/ubuntu2204/models/Qwen2.5-0.5B...
WARNING 03-28 11:38:20 [topk_topp_sampler.py:63] FlashInfer is not available. Falling back to the PyTorch-native implementation of top-p & top-k sampling. For the best performance, please install FlashInfer.
Loading safetensors checkpoint shards: 0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [01:50<00:00, 110.35s/it]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [01:50<00:00, 110.35s/it]
INFO 03-28 11:40:10 [loader.py:447] Loading weights took 110.39 seconds
INFO 03-28 11:40:10 [gpu_model_runner.py:1186] Model loading took 0.9267 GB and 110.720758 seconds
INFO 03-28 11:40:15 [backends.py:415] Using cache directory: /root/.cache/vllm/torch_compile_cache/872e86e684/rank_0_0 for vLLM's torch.compile
INFO 03-28 11:40:15 [backends.py:425] Dynamo bytecode transform time: 4.83 s
INFO 03-28 11:40:16 [backends.py:115] Directly load the compiled graph for shape None from the cache
INFO 03-28 11:40:19 [monitor.py:33] torch.compile takes 4.83 s in total
INFO 03-28 11:40:20 [kv_cache_utils.py:566] GPU KV cache size: 27,648 tokens
INFO 03-28 11:40:20 [kv_cache_utils.py:569] Maximum concurrency for 2,048 tokens per request: 13.50x
INFO 03-28 11:40:33 [gpu_model_runner.py:1534] Graph capturing finished in 13 secs, took 0.37 GiB
INFO 03-28 11:40:34 [core.py:151] init engine (profile, create kv cache, warmup model) took 23.24 seconds
WARNING 03-28 11:40:34 [config.py:1028] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`.
INFO 03-28 11:40:34 [serving_chat.py:115] Using default chat sampling params from model: {'max_tokens': 2048}
INFO 03-28 11:40:34 [serving_completion.py:61] Using default completion sampling params from model: {'max_tokens': 2048}
INFO 03-28 11:40:34 [api_server.py:1028] Starting vLLM API server on http://0.0.0.0:8000/
INFO 03-28 11:40:34 [launcher.py:26] Available routes are:
INFO 03-28 11:40:34 [launcher.py:34] Route: /openapi.json, Methods: HEAD, GET
INFO 03-28 11:40:34 [launcher.py:34] Route: /docs, Methods: HEAD, GET
INFO 03-28 11:40:34 [launcher.py:34] Route: /docs/oauth2-redirect, Methods: HEAD, GET
INFO 03-28 11:40:34 [launcher.py:34] Route: /redoc, Methods: HEAD, GET
INFO 03-28 11:40:34 [launcher.py:34] Route: /health, Methods: GET
INFO 03-28 11:40:34 [launcher.py:34] Route: /load, Methods: GET
INFO 03-28 11:40:34 [launcher.py:34] Route: /ping, Methods: POST, GET
INFO 03-28 11:40:34 [launcher.py:34] Route: /tokenize, Methods: POST
INFO 03-28 11:40:34 [launcher.py:34] Route: /detokenize, Methods: POST
INFO 03-28 11:40:34 [launcher.py:34] Route: /v1/models, Methods: GET
INFO 03-28 11:40:34 [launcher.py:34] Route: /version, Methods: GET
INFO 03-28 11:40:34 [launcher.py:34] Route: /v1/chat/completions, Methods: POST
INFO 03-28 11:40:34 [launcher.py:34] Route: /v1/completions, Methods: POST
INFO 03-28 11:40:34 [launcher.py:34] Route: /v1/embeddings, Methods: POST
INFO 03-28 11:40:34 [launcher.py:34] Route: /pooling, Methods: POST
INFO 03-28 11:40:34 [launcher.py:34] Route: /score, Methods: POST
INFO 03-28 11:40:34 [launcher.py:34] Route: /v1/score, Methods: POST
INFO 03-28 11:40:34 [launcher.py:34] Route: /v1/audio/transcriptions, Methods: POST
INFO 03-28 11:40:34 [launcher.py:34] Route: /rerank, Methods: POST
INFO 03-28 11:40:34 [launcher.py:34] Route: /v1/rerank, Methods: POST
INFO 03-28 11:40:34 [launcher.py:34] Route: /v2/rerank, Methods: POST
INFO 03-28 11:40:34 [launcher.py:34] Route: /invocations, Methods: POST
INFO: Started server process [11540]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO 03-28 11:40:44 [loggers.py:80] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
INFO 03-28 11:40:54 [loggers.py:80] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%At this point I think the server is up and running and can be requested. but i use curl command return Connection timed out
curl http://0.0.0.0:8000/v1/models
curl: (28) Failed to connect to 0.0.0.0 port 8000 after 133149 ms: Connection timed outThe port is indeed occupied
netstat -ano -p TCP
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name Timer
tcp 0 0 127.0.1.1:44742 0.0.0.0:* LISTEN 11746/python off (0.00/0/0)
tcp 0 0 127.0.1.1:45036 0.0.0.0:* LISTEN 11746/python off (0.00/0/0)
tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN 11671/python off (0.00/0/0)
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 152/systemd-resolve off (0.00/0/0)
tcp 0 0 127.0.1.1:46084 0.0.0.0:* LISTEN 11746/python off (0.00/0/0)
tcp 0 0 127.0.1.1:45644 0.0.0.0:* LISTEN 11746/python off (0.00/0/0)
tcp 0 0 10.255.255.254:53 0.0.0.0:* LISTEN - off (0.00/0/0)
tcp6 0 0 :::45836 :::* LISTEN 11746/python off (0.00/0/0)
tcp6 0 0 192.168.31.64:47319 192.168.31.64:45836 ESTABLISHED 11746/python off (0.00/0/0)
tcp6 0 0 192.168.31.64:45836 192.168.31.64:47319 ESTABLISHED 11746/python off (0.00/0/0)
udp 0 0 127.0.0.53:53 0.0.0.0:* 152/systemd-resolve off (0.00/0/0)
udp 0 0 10.255.255.254:53 0.0.0.0:* - off (0.00/0/0)
udp 0 0 127.0.0.1:323 0.0.0.0:* - off (0.00/0/0)
udp6 0 0 ::1:323 :::* - off (0.00/0/0)So what do I need to do can get vllm server return
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.