-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Open
Description
使用desktop执行“在桌面新建一个文件夹” 这种简单的指令,可以看到模型已经返回了
- 右击桌面
- 点击新建
这两个步骤,但是在实际执行过程中会出现不停循环点击桌面 后将侧边栏再点没 再右击桌面,无法执行到第二步
以下是日志
NFO 03-06 14:49:28 logger.py:39] Received request chatcmpl-c7e0ad4ff576497bb27e6a45e576261d: prompt: 'User: You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task.\n\n## Output Format\n\nThought: ...\nAction: ...\n\n\n## Action Space\nclick(start_box='[x1, y1, x2, y2]')\nleft_double(start_box='[x1, y1, x2, y2]')\nright_single(start_box='[x1, y1, x2, y2]')\ndrag(start_box='[x1, y1, x2, y2]', end_box='[x3, y3, x4, y4]')\nhotkey(key='')\ntype(content='') #If you want to submit your input, use "\n" at the end ofcontent.\nscroll(start_box='[x1, y1, x2, y2]', direction='down or up or right or left')\nwait() #Sleep for 5s and take a screenshot to check for any changes.\nfinished()\ncall_user() # Submit the task and call the user when the task is unsolvable, or when you need the user's help.\n\n## Note\n- Use Chinese inThoughtpart.\n- Write a small plan and finally summarize your next action (with its target element) in one sentence inThoughtpart.\n\n## User Instruction\n帮我在桌面新建一个文件夹<|im_end|>\nAssistant: 从当前页面来看,为了在桌面新建一个文件夹,我应该在桌面空白处点击右键,打开右键菜单。\nAction: right_single(start_box='(476,434)')<|im_end|>\nAssistant: 从当前页面来看,为了在桌面新建一个文件夹,我应该点击右键菜单中的"新建(W)"选项,以打开新建文件夹的子菜单。\nAction: click(start_box='(506,680)')<|im_end|>\nAssistant: 从当前页面来看,为了在桌面新建一个文件夹,我应该在桌面空白处点击右键,打开右键菜单。\nAction: right_single(start_box='(476,434)')<|im_end|>\nAssistant: 从当前页面来看,为了在桌面新建一个文件夹,我应该点击右键菜单中的"新建(W)"选项,以打开新建文件夹的子菜单。\nAction: click(start_box='(506,680)')<|im_end|>\nAssistant: 从当前页面来看,为了在桌面新建一个文件夹,我应该在桌面空白处点击右键,打开右键菜单。\nAction: right_single(start_box='(476,434)')<|im_end|>\nUser: <|vision_start|><|image_pad|><|vision_end|><|im_end|>\nAssistant: 从当前页面来看,为了在桌面新建一个文件夹,我应该点击右键菜单中的"新建(W)"选项,以打开新建文件夹的子菜单。\nAction: click(start_box='(506,680)')<|im_end|>\nUser: <|vision_start|><|image_pad|><|vision_end|><|im_end|>\nAssistant: 从当前页面来看,为了在桌面新建一个文件夹,我应该在桌面空白处点击右键,打开右键菜单。\nAction: right_single(start_box='(476,434)')<|im_end|>\nUser: <|vision_start|><|image_pad|><|vision_end|><|im_end|>\nAssistant: 从当前页面来看,为了在桌面新建一个文件夹,我应该点击右键菜单中的"新建(W)"选项,以打开新建文件夹的子菜单。\nAction: click(start_box='(506,680)')<|im_end|>\nUser: <|vision_start|><|image_pad|><|vision_end|><|im_end|>\nAssistant: 从当前页面来看,为了在桌面新建一个文件夹,我应该在桌面空白处点击右键,打开右键菜单。\nAction: right_single(start_box='(476,434)')<|im_end|>\nUser: <|vision_start|><|image_pad|><|vision_end|><|im_end|>\nAssistant: ', params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, top_p=1.0, top_k=-1, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=1000, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, guided_decoding=None), prompt_token_ids: None, lora_request: None, prompt_adapter_request: None.
INFO 03-06 14:49:28 engine.py:280] Added request chatcmpl-c7e0ad4ff576497bb27e6a45e576261d.
INFO 03-06 14:49:33 engine.py:298] Aborted request chatcmpl-c7e0ad4ff576497bb27e6a45e576261d.
INFO 03-06 14:49:33 metrics.py:455] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 878.2 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO 03-06 14:49:43 metrics.py:455] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
Metadata
Metadata
Assignees
Labels
No labels