[Bug]: At most 1 image(s) may be provided in one request; 400 Bad Request

[Chenky424]

Version

Agent-TARS-v1.0.0-alpha.7

Model

UI-TARS-7B-DPO

Deployment Method

Cloud

Issue Description

it just can carry out one-step demand, like 'open Chrome'. After finishing this, it always screenshotted again and then reported an error. Sometimes, it could not find the right location.

Error Logs

ERROR 04-09 17:23:46 [serving_chat.py:198] Error in preprocessing prompt inputs
ERROR 04-09 17:23:46 [serving_chat.py:198] Traceback (most recent call last):
ERROR 04-09 17:23:46 [serving_chat.py:198] File "/mnt/speech/ruiyu/miniconda3/envs/tars/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 181, in create_chat_completion
ERROR 04-09 17:23:46 [serving_chat.py:198] ) = await self._preprocess_chat(
ERROR 04-09 17:23:46 [serving_chat.py:198] File "/mnt/speech/ruiyu/miniconda3/envs/tars/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_engine.py", line 391, in _preprocess_chat
ERROR 04-09 17:23:46 [serving_chat.py:198] conversation, mm_data_future = parse_chat_messages_futures(
ERROR 04-09 17:23:46 [serving_chat.py:198] File "/mnt/speech/ruiyu/miniconda3/envs/tars/lib/python3.10/site-packages/vllm/entrypoints/chat_utils.py", line 1139, in parse_chat_messages_futures
ERROR 04-09 17:23:46 [serving_chat.py:198] sub_messages = _parse_chat_message_content(
ERROR 04-09 17:23:46 [serving_chat.py:198] File "/mnt/speech/ruiyu/miniconda3/envs/tars/lib/python3.10/site-packages/vllm/entrypoints/chat_utils.py", line 1067, in _parse_chat_message_content
ERROR 04-09 17:23:46 [serving_chat.py:198] result = _parse_chat_message_content_parts(
ERROR 04-09 17:23:46 [serving_chat.py:198] File "/mnt/speech/ruiyu/miniconda3/envs/tars/lib/python3.10/site-packages/vllm/entrypoints/chat_utils.py", line 967, in _parse_chat_message_content_parts
ERROR 04-09 17:23:46 [serving_chat.py:198] parse_res = _parse_chat_message_content_part(
ERROR 04-09 17:23:46 [serving_chat.py:198] File "/mnt/speech/ruiyu/miniconda3/envs/tars/lib/python3.10/site-packages/vllm/entrypoints/chat_utils.py", line 1024, in _parse_chat_message_content_part
ERROR 04-09 17:23:46 [serving_chat.py:198] mm_parser.parse_image(str_content)
ERROR 04-09 17:23:46 [serving_chat.py:198] File "/mnt/speech/ruiyu/miniconda3/envs/tars/lib/python3.10/site-packages/vllm/entrypoints/chat_utils.py", line 725, in parse_image
ERROR 04-09 17:23:46 [serving_chat.py:198] placeholder = self._tracker.add("image", image_coro)
ERROR 04-09 17:23:46 [serving_chat.py:198] File "/mnt/speech/ruiyu/miniconda3/envs/tars/lib/python3.10/site-packages/vllm/entrypoints/chat_utils.py", line 548, in add
ERROR 04-09 17:23:46 [serving_chat.py:198] raise ValueError(
ERROR 04-09 17:23:46 [serving_chat.py:198] ValueError: At most 1 image(s) may be provided in one request.
INFO: 127.0.0.1:32910 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request

[yytdfc]

+1 same issue

[ulivz]

Thank you for your detailed feedback and analysis.

The UI-TARS-7B-DPO model is currently designed to work only with UI-TARS Desktop and supports GUI Agent-related capabilities. It does not yet support integration with Agent-TARS, which is likely causing the issues you observed.

Some clarifications

These are two applications:

• UI TARS Desktop is our first GUI Agent application focused on controlling computers, with its latest version being v0.0.8, which supports both Mac and Windows.

• Agent TARS App is our new application focused on browser agents, with its latest version being Agent-TARS-v1.0.0-alpha.7. Since it is still in the technical preview stage, it currently supports only Mac.

Suggestions

1. Use a model compatible with Agent-TARS: [Agent TARS] Thank You for Your Support + Updates on Model Compatibility #377

2. Future Development: Long-term, we plan to make UI-TARS models, integrate with Agent-TARS. This is under technical research, so stay tuned for updates.

[zhurunhua]

UI-TARS-1.5-7B same issue