Skip to content

dsh-win-computer-use 0.1.0

Choose a tag to compare

@wwwort wwwort released this 19 Sep 11:31
· 7 commits to main since this release

Windows-native computer use for DeepSeek Harness.

What it saves (measured, not estimated)

DeepSeek Harness records usage on every assistant message, so every number here is a
reading from its own session log.

One tool call per action This plugin
An 8-action task 8+ model requests 1 model request
The conversation, re-sent 8 times once
Measured: one request re-sent — 185,404 tokens for the 8-step batch call; per-request average in the same long session 437,542 (peak 473,134)
Same task, in context tokens 8 × 437k ≈ 3.5M ≈ 437k → ~3.06M less (88%)
Engine startup ~800 ms per call ~800 ms once, then 40–100 ms per action
Seeing the result screenshot, then a second read_image request screenshot inline in the same call

The lever is the round trip, not the schema. Straight about the schema: merging eight tools into
two made that block bigger — 4,149 → 6,977 characters — because computer's step schema
documents 41 fields. It is a fixed prefix served from the prompt cache, and growing it is a deliberate
trade: the field documentation is what lets the model write a whole steps array correctly on the
first attempt, and one avoided retry is another full context re-send.

When to delegate (two experiments, measured)

An operation can be run by a subagent instead of by the conversation driving it. The whole
question is how many turns the main agent would have needed, and both cases were measured.

A task the main agent can specify without looking:

Arm Cost
Main agent drives it directly (one batch call) 522,925 — 1 request
Delegated to a subagent 525,007 + 99,092 (5 child requests) = 624,099

→ delegating cost 19% more; "operations always go to a subagent" is wrong.

A cold, multi-turn task (the window shows a random code, so nothing can be typed before it is
read off the screen):

Arm Cost
Main agent drives it directly 559,092 + 560,636 + 562,535 = 1,682,263 — 3 requests
Delegated to a subagent 565,459 + 98,535 (5 child requests) = 663,994

→ delegating saved 61%. Both arms verified themselves with no focus change.

Main agent ≈560,000 tokens/request; child ≈20,000 — a factor of 28. The child's side barely grows
with complexity while the main agent's grows linearly, so the saving scales:

Turns the main agent would have needed Main agent Delegated Saving
1 0.56M 0.66M −15% (don't)
2 1.12M 0.64M 43%
3 1.68M 0.66M 61% (measured)
5 2.80M 0.70M 75%
10 5.60M 0.80M 86%
20 11.2M 0.99M 91%

The more complex the task, the more it saves — asymptotically ~96%.

Routing rule: (1) whole steps array writable without looking → do it yourself in one call;
(2) otherwise → delegate the whole thing to a subagent and don't scout it first.

Reuse, measured: one durable background child did two tasks for 120,803 child tokens, its context
growing only 18,496 → 21,554. It still cannot pay for itself — a background child costs the parent one
extra turn per result, ~14× the two tasks' entire child budget. Cheap turn or reusable operator: pick
one.

The protocol ships as the computer-operator skill, costing one catalog line until loaded.

Tools (two, deliberately)

  • computer — takes a steps array and runs find / click / type / key / wait / shot /
    windows / uia / mouse / focus / clipboard / process / display / sleep inside one
    engine request
    . Steps address controls by name (as: "ref", valid within the same call). A shot
    as the last step returns the picture inline.
  • computer_shot — offscreen window capture (PrintWindow), no activation.

Focus-free by default

Input prefers UI Automation patterns, then Win32 messages straight to the control; screenshots use
PrintWindow, which captures a window that is covered. Physical input is the last resort and restores
the previous foreground window and cursor afterwards. mode: "background" refuses rather than
silently taking your screen.

Install

dsh plugin --profile web add github:wwwort/dsh-win-computer-use

Requires Windows and Windows PowerShell 5.1.