How do you get llmfit to see that vLLM is running?

[akulbe]

I've got vLLM running in a tmux session, and I open a new pane with llmfit open, and try to download a model, and it says vllm is required. I'm not sure what I'm missing.

[AlexsJones]

Good question, two things are going on here.

llmfit doesn't look for a vllm binary or process; it detects vLLM by probing the OpenAI-compatible endpoint at http://localhost:8000/v1/models. If your server is up on the default port, the tmux pane doesn't matter and it should show as available. If you started it with --port (or it's on another host), point llmfit at it with the VLLM_HOST env var, e.g. VLLM_HOST=localhost:8080 llmfit.

Separately: downloading a model through llmfit isn't supported for vLLM even when it's detected. vLLM loads its model at server startup, so the flow is to restart it with the model you want (vllm serve <model>), and llmfit will then list it as installed. So if your goal was "download via llmfit, run in vLLM", that error is expected; detection above should still show the runtime as available.

If it still shows as not detected with the server on port 8000, paste the output of curl -s localhost:8000/v1/models and we'll dig in.