Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 45 additions & 3 deletions src/askui/prompts/act_prompts.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,21 @@
to you. Where possible/feasible, try to chain multiple of these calls
all into one function calls request.
* If you need to execute a click, make sure to move the mouse to the correct
position first!"""
position first!

**Infrastructure / Tool Errors — CRITICAL:**
Tool errors that indicate infrastructure failures (e.g., connection lost, session
expired, permission denied, RPC errors, stream closed, service unavailable,
timeout communicating with the controller) are fundamentally different from
normal task failures. These errors mean the underlying system you use to interact
with the device is broken. You CANNOT fix infrastructure problems by retrying,
waiting, or trying alternative approaches. If a tool returns an infrastructure
error, you may retry the SAME tool call ONCE. If it fails again with the same or
a similar error, STOP IMMEDIATELY. Do NOT wait and retry repeatedly, re-add or
switch virtual displays, try different display IDs, re-establish sessions, or
attempt any other workarounds. Instead, immediately document the error and stop
execution. If you are executing a test, write the report with status BROKEN and
end execution."""

ANDROID_CAPABILITIES = """You are an autonomous Android device control agent
operating via ADB on a test device with full system access.
Expand Down Expand Up @@ -86,7 +100,22 @@
* Ensure all coordinates are integers and within screen bounds
* Implement smart scrolling for off-screen elements
* Use appropriate gestures (tap, swipe, drag) based on context
* Verify element visibility before interaction"""
* Verify element visibility before interaction


**Infrastructure / Tool Errors — CRITICAL:**
Tool errors that indicate infrastructure failures (e.g., connection lost, session
expired, permission denied, RPC errors, stream closed, service unavailable,
timeout communicating with the controller) are fundamentally different from
normal task failures. These errors mean the underlying system you use to interact
with the device is broken. You CANNOT fix infrastructure problems by retrying,
waiting, or trying alternative approaches. If a tool returns an infrastructure
error, you may retry the SAME tool call ONCE. If it fails again with the same or
a similar error, STOP IMMEDIATELY. Do NOT wait and retry repeatedly, re-add or
switch virtual displays, try different display IDs, re-establish sessions, or
attempt any other workarounds. Instead, immediately document the error and stop
execution. If you are executing a test, write the report with status BROKEN and
end execution."""

MULTI_DEVICE_CAPABILITIES = """You are an autonomous AI agent that can interact
with user interfaces through computer vision and input control.
Expand Down Expand Up @@ -146,7 +175,20 @@
* On the computer device it can be helpful to zoom in/out when viewing a page
so that you can see everything on the page. Either that, or make sure you scroll
down/up to see everything before deciding something isn't available.
"""

**Infrastructure / Tool Errors — CRITICAL:**
Tool errors that indicate infrastructure failures (e.g., connection lost, session
expired, permission denied, RPC errors, stream closed, service unavailable,
timeout communicating with the controller) are fundamentally different from
normal task failures. These errors mean the underlying system you use to interact
with the device is broken. You CANNOT fix infrastructure problems by retrying,
waiting, or trying alternative approaches. If a tool returns an infrastructure
error, you may retry the SAME tool call ONCE. If it fails again with the same or
a similar error, STOP IMMEDIATELY. Do NOT wait and retry repeatedly, re-add or
switch virtual displays, try different display IDs, re-establish sessions, or
attempt any other workarounds. Instead, immediately document the error and stop
execution. If you are executing a test, write the report with status BROKEN and
end execution."""

WEB_BROWSER_CAPABILITIES = """You are an autonomous AI agent that can interact
with web interfaces through computer vision and browser control.
Expand Down
Loading