Background Targeting

Background / Unfocused Window Targeting

A first-class capability: drive a specific window without bringing it to the foreground or stealing the user's focus.

mouse_click, type_text and screenshot all accept hwnd or window_title. When set, input is delivered to that exact window via OS messages, and screenshot captures the window even if it's occluded, minimized, or on an off-screen desktop.

How it works

	Windows	Linux
Click	PostMessage WM_BUTTON to the deepest child control at the point	xdotool mousemove/click --window
Type	WM_CHAR to the focused control / WM_SETTEXT to a control	xdotool type --window (XSendEvent)
Keys	WM_KEYDOWN/WM_KEYUP	xdotool key --window
Capture	PrintWindow (PW_RENDERFULLCONTENT)	ImageMagick import -window

On Windows hwnd is an HWND; on Linux it is an X11 window id. Background click coordinates are client coordinates of the target window.

Standard workflow

1. Find the top-level window:

list_windows { "title_filter": "Notepad" }

2. Find the specific control and its rect:

list_child_windows { "window_title": "Notepad" }

3. Drive it without focus (most reliable text entry first):

win_set_control_text { "hwnd": 23924320, "text": "no focus needed" }

mouse_click { "window_title": "Notepad", "x": 200, "y": 120 }

win_send_keys { "window_title": "Notepad", "keys": ["ctrl", "s"] }

4. See the result without raising the window:

screenshot { "window_title": "Notepad" }

Limitations (important)

• Some apps ignore synthetic input — those that read raw input / DirectInput or check physical key state (GetAsyncKeyState), and xterm with its default allowSendEvents: false. For those:
  • Windows → use win_set_control_text, or AutoHotkey ahk_control_send (ControlSend is more robust), or focus the window with window_action.
  • Linux → focus the window, or relaunch the app permitting send-events.
• Message-based modifier combos (Ctrl+C) are unreliable via win_send_keys; prefer win_set_control_text for text, or AHK for hotkeys.
• PrintWindow renders black on a few GPU-exclusive surfaces — fall back to a region screenshot.

See also Headless GUI (drive apps that aren't on the visible desktop at all).