Serving local models in UI TARS desktop

[daphnechen]

I am trying to use a model that I have downloaded and stored locally on my device (UI TARS 7B DPO) but after running the command in the readme:

python -m vllm.entrypoints.openai.api_server --served-model-name ui-tars --model <path to your model>

and specifying the path to the UI TARS 7B DPO model, then updating the endpoint and model name in the UI settings, I am still not able to get it to work. The UI outputs the error "ERROR: 400 status code (no body)". Any idea how I can fix this?

[Dinnkunwork]

Yeah, me too. Also, somehow my model can't type even though the log says it should.

[AHEADer]

Can you share your environment setting for using vLLM? May I know the vLLM version so I can try to reproduce this error.

[alfredwallace7]

Same issue here where requests to /v1/chat/completions return a 400 Bad Request error with no response body. Interestingly, the first action (screenshot/click) appears to work correctly then the logs indicate an error in preprocessing prompt inputs, specifically in _parse_chat_message_content_part, which results in a ValueError: At most 1 image(s) may be provided in one request.

The error appears to originate from the following files and lines in the vLLM package:

serving_chat.py (line 181)
serving_engine.py (line 454)
chat_utils.py (lines 961, 889, 791, 849, 583, 464)

Environment:

OS: WSL2 (Debian Bookworm)
Model: bytedance-research/UI-TARS-7B-DPO
Python: 3.11.2
vLLM: 0.6.6
Transformers: @ git+https://github.com/huggingface/transformers.git@336dc69d63d56f232a183a3e7f52790429b871ef
Torch: 2.5.1+cu124
Torchvision: 0.20.1+cu124
Accelerate: 1.4.0
Flash Attention: 2.7.4.post1

Command used to start vLLM API server:

python -m vllm.entrypoints.openai.api_server --served-model-name ui-tars --model bytedance-research/UI-TARS-7B-DPO --max-model-len 14000