dsh-win-computer-use 0.1.1

Windows-native computer use for DeepSeek Harness. 0.1.1 closes the element-tree gap
on both sides: a whole chat round now runs with no element tree at all.

Reading is cheaper than looking

Text content is read as text, not captured as pixels:

{"op": "read", "window": "ChatGPT", "tail": 2000}                     // newest 2000 chars
{"op": "wait", "window": "ChatGPT", "state": "text_stable", "tail": 2000,
 "absent": "正在回应", "contains": "ChatGPT 说"}

Measured on the ChatGPT desktop app: one long reply came back as 2,200 characters of
quotable text in a single ~280 ms call; the whole page is 25,722 characters through the
same path. Document reading order puts the newest content last and the app's own chrome
first, so a tail read is clean by construction — and it never touches the sidebar.

It is also more reliable than element search in browsers. At one point the ChatGPT
window's UIA element tree had collapsed to 13 nodes (caption buttons and empty panes,
the entire page gone) while read still returned the full page text. A find returning
nothing in a Chromium app does not mean the control is absent.

Writing without the element tree

type and key take an x/y point and click it first, because activating a window
does not give its input box keyboard focus — measured: keystrokes after a plain activation
were dropped, Ctrl+A selected nothing, and Enter went to whatever the app itself had
focused. That last one is how a "send" silently does nothing.

{"steps": [
  {"op": "type", "window": "ChatGPT", "mode": "physical", "x": 1436, "y": 1205,
   "text": "...", "verify": true},
  {"op": "key", "keys": "enter", "window": "ChatGPT"},
  {"op": "wait", "window": "ChatGPT", "state": "text_stable", "absent": "正在回应", "tail": 2000}
]}

Long text can go "via": "clipboard" — one clipboard write plus Ctrl+V instead of
thousands of SendInput records (the previous clipboard text is restored).

Five traps, each one measured

1. focus activates a window but does not focus its control.
2. The foreground is now returned once per call, not per step: restoring it between
   steps of the same batch steals the target app's focus and the next Enter lands nowhere
   (reported as foregroundRestored; restore: false opts out).
3. Pure "the text stopped changing" settles on a "still working" placeholder — hence the
   absent / contains guards.
4. Chromium's element tree can collapse; text keeps working.
5. contenteditable text updates lazily, so an immediate read-back can show a stale value.

Also in this release

• One batch tool runs a whole find / click / type / key / wait / read / shot / … sequence
  in one model request; a screenshot as the last step returns the picture inline.
• Measured cost of a round trip: an 8-action task is 8+ requests or 1; the per-request
  context averaged 437,542 tokens in a long session, so one batch call saves ~3.06M context
  tokens (88%) on that shape.
• Routing by complexity, measured both ways: a task the agent can specify without looking
  should not be delegated (−15%), while a cold multi-turn task saves 61% delegated
  (86% at ten turns). The protocol ships as the computer-operator skill.
• Focus-free by default: UI Automation patterns, Win32 messages, offscreen PrintWindow
  capture. mode: "background" refuses rather than silently taking your screen.
• Warm engine over loopback TCP: Add-Type is paid once (~800 ms/call → 40–100 ms/action).

Install

dsh plugin --profile web add github:wwwort/dsh-win-computer-use

Requires Windows and Windows PowerShell 5.1.

dsh-win-computer-use 0.1.0

Windows-native computer use for DeepSeek Harness.

What it saves (measured, not estimated)

DeepSeek Harness records usage on every assistant message, so every number here is a
reading from its own session log.

	One tool call per action	This plugin
An 8-action task	8+ model requests	1 model request
The conversation, re-sent	8 times	once
Measured: one request re-sent	—	185,404 tokens for the 8-step batch call; per-request average in the same long session 437,542 (peak 473,134)
Same task, in context tokens	8 × 437k ≈ 3.5M	≈ 437k → ~3.06M less (88%)
Engine startup	~800 ms per call	~800 ms once, then 40–100 ms per action
Seeing the result	screenshot, then a second read_image request	screenshot inline in the same call

The lever is the round trip, not the schema. Straight about the schema: merging eight tools into
two made that block bigger — 4,149 → 6,977 characters — because computer's step schema
documents 41 fields. It is a fixed prefix served from the prompt cache, and growing it is a deliberate
trade: the field documentation is what lets the model write a whole steps array correctly on the
first attempt, and one avoided retry is another full context re-send.

When to delegate (two experiments, measured)

An operation can be run by a subagent instead of by the conversation driving it. The whole
question is how many turns the main agent would have needed, and both cases were measured.

A task the main agent can specify without looking:

Arm	Cost
Main agent drives it directly (one batch call)	522,925 — 1 request
Delegated to a subagent	525,007 + 99,092 (5 child requests) = 624,099

→ delegating cost 19% more; "operations always go to a subagent" is wrong.

A cold, multi-turn task (the window shows a random code, so nothing can be typed before it is
read off the screen):

Arm	Cost
Main agent drives it directly	559,092 + 560,636 + 562,535 = 1,682,263 — 3 requests
Delegated to a subagent	565,459 + 98,535 (5 child requests) = 663,994

→ delegating saved 61%. Both arms verified themselves with no focus change.

Main agent ≈560,000 tokens/request; child ≈20,000 — a factor of 28. The child's side barely grows
with complexity while the main agent's grows linearly, so the saving scales:

Turns the main agent would have needed	Main agent	Delegated	Saving
1	0.56M	0.66M	−15% (don't)
2	1.12M	0.64M	43%
3	1.68M	0.66M	61% (measured)
5	2.80M	0.70M	75%
10	5.60M	0.80M	86%
20	11.2M	0.99M	91%

The more complex the task, the more it saves — asymptotically ~96%.

Routing rule: (1) whole steps array writable without looking → do it yourself in one call;
(2) otherwise → delegate the whole thing to a subagent and don't scout it first.

Reuse, measured: one durable background child did two tasks for 120,803 child tokens, its context
growing only 18,496 → 21,554. It still cannot pay for itself — a background child costs the parent one
extra turn per result, ~14× the two tasks' entire child budget. Cheap turn or reusable operator: pick
one.

The protocol ships as the computer-operator skill, costing one catalog line until loaded.

Tools (two, deliberately)

• computer — takes a steps array and runs find / click / type / key / wait / shot /
  windows / uia / mouse / focus / clipboard / process / display / sleep inside one
  engine request. Steps address controls by name (as: "ref", valid within the same call). A shot
  as the last step returns the picture inline.
• computer_shot — offscreen window capture (PrintWindow), no activation.

Focus-free by default

Input prefers UI Automation patterns, then Win32 messages straight to the control; screenshots use
PrintWindow, which captures a window that is covered. Physical input is the last resort and restores
the previous foreground window and cursor afterwards. mode: "background" refuses rather than
silently taking your screen.

Install

dsh plugin --profile web add github:wwwort/dsh-win-computer-use

Requires Windows and Windows PowerShell 5.1.