Repository navigation
0.113.0: Participant impressions, mission and persona warnings, local desktop capacity, key status, update notice
humanish 0.113.0 asks computer-use participants what they thought at the end of a session, warns
when a study scripts its participants, and says how many local desktops fit. An OpenAI or Codex
participant ends with up to six first-person impressions: what looked unclear, unfinished or
untrustworthy, what they liked, what they expected and did not find, and where the screen differs
from how they do the same task in their own work or life. The trace keeps them as impressions,
the Observer shows them under "What they said at the end", and analyses at prompt
study-evidence-8 can cite them. study check and run warn when a mission reads like a list of
UI steps or a participant's persona has no background, and the starter personas init writes now
lead with a background, which changes their participant prompt. runtime status and
doctor --study report how many participant desktops the local runtime holds, and a live local
study that needs more than a Mac's humanish VM holds is refused before any desktop starts.
humanish keys lists each provider key with its source and never its value, and humanish says at
most once a day, at a terminal, when a newer version is out. Live shared-world output and older
terminal and scripted runs use plain participant captions.
This is a minor release because four defaults change. An OpenAI participant that ends the session
itself makes one more model request, for its impressions, which the run's cost includes. The
starter personas init writes give their participants a different prompt. A live local study on a
Mac that needs more desktops than the VM holds is refused where 0.112.0 started it. A command run
at a terminal starts a registry request at most once a day for the update notice. It removes and
renames no command, flag, study field, bundle field or library export, and the package exports the
same 13 values and 31 types as 0.112.0. It adds the trace field impressions, the debrief
trigger participant_end, the optional provider method requestImpressions and turn field
impressions, the error codes HUMANISH_COMPUTER_USE_LOCAL_CAPACITY_EXCEEDED and
HUMANISH_COMPUTER_USE_DESKTOP_OUT_OF_MEMORY, runtime setup --memory and --cpus,
keys status, and capacity in runtime status --json. Fourteen outputs change for the same
input, which a script could notice:
| # | 0.112.0 | 0.113.0 | What to do |
|---|---|---|---|
| 1 | An OpenAI computer-use participant that ended the session itself made no closing request. Its trace had no debrief, no impressions, and counts.debriefCalls 0. |
One more read-only request, with no tools, asks for its impressions only. The trace gains debrief (trigger: "participant_end", status, reason), counts.debriefCalls 1, one more tokenUsage.turns entry, impressions as {status: "collected", items: [{kind, text, messageId}]} or {status: "not_collected", reason}, and one message item per impression, which counts.messages does not count. The participant's summary, reason and outcome come from its own last message, as before. A closing report after a harness stop asks for impressions too, and the trace keeps them in impressions, out of debrief.report. A Claude local-agent participant records not_collected. (#1650) |
Read impressions and debrief as optional. Count a participant's turns from counts.turns. |
| 2 | Analyses were written at prompt study-evidence-7. |
They are written at study-evidence-8, and each impression is an evidence item, Impression (<kind>): <text>, that findings and design findings cite. 0.112.0's review reads a study-evidence-8 analysis and prints it as 0.113.0 does. (#1650) |
Nothing. |
| 3 | An OpenAI closing request could use at most 1024 output tokens. | Up to 3072, or the study's actor.maxOutputTokens when that is lower. (#1650) |
Nothing. |
| 4 | init wrote the starter personas synthetic-new-user and skeptical-power-user with patience and technical_confidence traits and no background. |
They lead with a short fictional background; only accessibility_needs: keyboard_first stays a trait. Their participants' prompt and the plan's prompt# digests change. init in an existing project changes no file, so a project made by an earlier release keeps the old personas. (#1656) |
Add a background to each persona study check names. |
| 5 | study check and run printed nothing about a mission written as UI steps or a persona without a background. |
They print warning: actor.mission reads like a script, which can hide where a participant would get lost. Matched lines: with up to three quoted lines and a situation-style example, or warning: Participant <id> has no persona background because persona <persona> has no readable background. .... study check --json lists them in warnings[], and a run, including one from the library's runStudy, records each as a study.warning event. Neither refuses. A project made by 0.112.0 gets the persona warning on try-live. Scripted and terminal studies and tasks[].success are not checked. (#1656) |
Describe a situation in the mission and keep criteria in tasks[].success. Expect more study.warning events in run.json. |
| 6 | study check listed credentials among what the metadata check did not cover. --reachability prepared-host failed every study, and public-preview or sandbox-loopback failed a local study, each with exit 2 and HUMANISH_STUDY_PREFLIGHT_UNSUPPORTED_ROUTE. |
Every study check adds a row, not checked this machine: ... Run npx humanish doctor --study <study> to check them. A valid study in those modes passes, exit 0, with a not checked reachability row naming why. These rows carry checked: false in JSON. (#1658) |
Read a row with checked: false as not checked, and run doctor --study for it. |
| 7 | A live local study on a Mac that ran more desktops at once than the Lima VM held started, and the desktops past its memory crashed without a message. A new VM got 8 GiB and 6 CPUs. | It is refused before the run directory, any desktop or model call, with HUMANISH_COMPUTER_USE_LOCAL_CAPACITY_EXCEEDED, a message saying how many it runs and how many fit, and what to do. On Linux the same text is a warning and the run starts. A desktop Docker kills for memory fails with HUMANISH_COMPUTER_USE_DESKTOP_OUT_OF_MEMORY, in the participant's error, the run's error and review.md. A new VM gets 13 GiB and 8 CPUs, capped at half the Mac. (#1658) |
Lower execution.concurrency, run runtime setup --memory <GiB>, or run on cloud desktops (subject.source: local-tree with execution.target: e2b-desktop). |
| 8 | runtime status printed whether the runtime is installed. runtime setup had no --memory or --cpus. |
runtime status adds the memory, CPUs, each desktop's 3 GiB and 2 CPUs, and how many desktops fit; --json carries them as capacity (host, memoryGiB, cpus, reservedMemoryGiB, perDesktop, desktops, sharesCpus). doctor --study on a local study adds a local desktop capacity row, and starts with a next: line when the runtime is not set up. runtime setup --memory --cpus resizes the Mac's VM and exits 2 on Linux. (#1658) |
Read capacity.desktops for how many desktops fit. |
| 9 | --rerun-failed-from refusals read "source run is dry-run; rerun selection only applies to live CUA fan-out evidence." and "source run is not a CUA fan-out run." |
"Run is a dry run: no participant ran in it, so none can be rerun. ...", "Run is not a live computer-use run with more than one participant, so it has no participants to rerun.", each followed by the watch and run commands for the whole study. The code is unchanged. (#1658) |
Match a refusal by error.code, HUMANISH_COMPUTER_USE_RERUN_INVALID. |
| 10 | humanish keys and keys --json printed the usage block on stderr and exited 1. keys set with no key named exited 1 with "error: missing required argument 'vendor-or-name'". |
keys (or keys status) prints each provider key humanish uses with its source or the command that adds it, and exits 0; --json is humanish.keys-result.v1 with action: "status", keys[] (name, use, source, hint) and otherStored. No value is printed. keys set with no key named asks for each missing key at a terminal; without one it exits 2 and names the keys set e2b --stdin form. A missing AGENTMAIL_API_KEY points at humanish keys set agentmail. (#1660) |
Read key status from keys --json. From a script, name the key and pass --stdin. |
| 11 | Every command's help listed help [command] under Commands. |
It is not listed. humanish help <command> and humanish <command> help still work. (#1660) |
Nothing. |
| 12 | humanish never checked for a newer version. | After a command at a terminal, at most once a day, one stderr line when a newer version is known: "humanish is out of date; the latest is . Run npx humanish@latest to use it." A detached process asks the npm registry for humanish's dist-tags at most once a day, with no identifying data and a 3 s timeout, and writes update-check.json beside telemetry.json. doctor shows a humanish version note once a check has succeeded. It is off in CI, without a terminal, for --json, for study participants, in humanish's own checkout, and with DO_NOT_TRACK, HUMANISH_TELEMETRY_DISABLED or HUMANISH_NO_UPDATE_CHECK=1. (#1660) |
Set HUMANISH_NO_UPDATE_CHECK=1 to turn it off. |
| 13 | The TUI footer read ? keys and c connections. |
? shortcuts and c keys and accounts; a narrow terminal drops the word "move" from the legend. (#1660) |
Nothing. |
| 14 | The end of a live shared-world run printed topology: shared-world/concurrent (2 personas, concurrency 2), plane: <repo>@<commit>, overlap: proven (this run only; no scale or adoption claim) and persona <id> (<persona>): passed (goal_satisfied). observer-data.json streams[].embed.title read "Shared plane, persona ". Terminal and scripted runs recorded before 0.112.0 kept captions such as Terminal agent · <study> in the Observer and the TUI. |
participants: 2, up to 2 at once, app: <repo>@<commit>, "Participants used the app at the same time (observed in this run only).", and Weekly planner: passed. The embed title is the participant's caption and screenshot mode, such as "Weekly planner (raw)". Older terminal and scripted runs take their persona's name, with ", phone" for a scripted mobile surface. Verdicts, statuses, error codes, schema ids and JSON field names are unchanged. (#1655) |
Read outcomes from run --json or run.json. |
Added
- Computer-use participants give impressions at the end of a session: up to six first-person
notes on what looked unclear, unfinished or untrustworthy, what they liked, what they expected
and did not find, and where the screen differs from how their persona does the same task in
their own work or life. The OpenAI closing report and the Codex participant's final account ask
for them. When an OpenAI participant ends the session itself, one more request asks for its
impressions only; it adds one model request per participant and leaves the participant's
summary, reason and outcome as they were. The trace'simpressionskeeps them, each recorded as
a message the analysis can quote and cite, or says why none were collected, for example when a
Claude local-agent participant ends the session itself or the request fails. The Observer shows
them under "What they said at the end", grouped by kind. An analysis can cite an impression as a
participant statement, and an impression with a capture of that screen as a design finding. A
feedback draft for a finding that cites one quotes it. The analysis prompt is
study-evidence-8(rows 1 and 2). (#1650) humanish study checkandhumanish runwarn when a computer-use or shared-world mission
scripts UI steps (numbered or bulleted steps, a chain of clicks and typing, quoted control
labels, CSS selectors, or URLs other than the subject's) or a participant has no persona
background. A mission warning quotes up to three matched lines and suggests a situation-style
brief. Runs keep both warnings in their evidence bundles. The personasinitwrites now lead
with a short fictional background (rows 4 and 5). (#1656)humanish keyswith no subcommand, orhumanish keys status, lists the provider keys humanish
uses (E2B_API_KEY,OPENAI_API_KEY,GH_TOKEN,AGENTMAIL_API_KEY), each with the source
that supplies it (environment, project file, user store,gh auth token,e2b auth login) or
the command that adds it. It never prints a value. It used to print the usage block (row 10).
(#1660)humanish keys setwith no key named, run at a terminal, asks for each missing provider key in
turn with hidden input. Press Enter to skip a key, or Ctrl+C to stop and keep the keys already
stored. Without a terminal it still needs a key named, and its error names the--stdinform
(row 10). (#1660)- humanish tells you when a newer version is out. At most once a day, after a command run at a
terminal, one line on stderr names your version, the latest one andnpx humanish@latest. The
registry request runs in a background process, carries no identifying data and never delays or
fails a command. It is off in CI, without a terminal, for--jsonoutput, and with
DO_NOT_TRACK=1,HUMANISH_TELEMETRY_DISABLED=1orHUMANISH_NO_UPDATE_CHECK=1.
humanish doctorshows the version the last check found (row 12). (#1660) humanish runtime setup --memory <GiB> --cpus <n>sizes the Mac's humanish Lima VM. An
existing VM keeps its size unless you pass them; with them, setup stops the VM, resizes it and
starts it again, and refuses while participant desktops are running in it. (#1658)- A live local study that runs more participant desktops at once than the Lima VM holds is
refused before any desktop or model call withHUMANISH_COMPUTER_USE_LOCAL_CAPACITY_EXCEEDED.
The message says how many it runs, how many fit and why, then what to do: run it on cloud
desktops (subject.source: local-treewithexecution.target: e2b-desktop), give the VM more
room withruntime setup --memory, or lowerexecution.concurrency. Memory sets the limit:
desktops that share fewer than 2 CPUs each only run slower, andruntime statussays so. On
Linux the same shortfall prints a warning and the run starts (row 7). (#1658) humanish runtime status(and--json, ascapacity) reports the VM's or this machine's
memory and CPUs, each desktop's 3 GiB, and how many desktops fit.humanish doctor --studyon
a local study adds a local desktop capacity row that says whether the study fits (row 8).
(#1658)- When Docker kills a local participant's desktop for memory, the participant's error, the run's
error andreview.mdsay the desktop ran out of memory and what to change, with the code
HUMANISH_COMPUTER_USE_DESKTOP_OUT_OF_MEMORY(row 7). (#1658)
Changed
- OpenAI computer-use closing requests may use up to 3072 output tokens, reasoning included, or
the study'sactor.maxOutputTokenswhen that is lower. They used at most 1024, which a closing
report with impressions could exceed, losing the whole report (row 3). (#1650) - A new humanish Lima VM gets room for four participant desktops plus 1 GiB for itself (13 GiB
and 8 CPUs), capped at half the Mac's memory and CPUs. It got 8 GiB and 6 CPUs on every Mac,
which holds two desktops (row 7). (#1658) humanish doctor --studyon a local study whose runtime is not set up starts with anext:
line naming the step to take, such ashumanish runtime setup, and shows a runtime that will
download on first use as a note (row 8). (#1658)humanish study checkpasses a valid study in a--reachabilitymode it cannot apply:
prepared-host, which has no CLI check, andpublic-previeworsandbox-loopbackon a local
study. Those rows, and a newthis machinerow, are marked not checked and name
humanish doctor --study <study>. The JSON rows carrychecked: false(row 6). (#1658)--rerun-failed-fromrefusals name the run they read, say why it cannot be rerun (no runs yet,
a dry run, a run with one participant, or no failed participant) and give thewatchandrun
commands for the whole study (row 9). (#1658)helpis no longer listed under Commands in any command's help.humanish help <command>and
humanish <command> helpstill work (row 11). (#1660)- The TUI footer says
? shortcutswhere it said? keys, andc keys and accountswhere it
saidc connections. On a narrow terminal the legend drops the word "move" so it stays on one
line (row 13). (#1660) - A missing
AGENTMAIL_API_KEYnow points athumanish keys set agentmail(row 10). (#1660) - The packed skill opens its study-authoring section with the mission guidance and a scripted and
a situation-style example side by side, and the study-files reference lists both warnings. The
local browser docs say how many participants fit, and theruntimereference covers
--memoryand--cpus. (#1656, #1658)
Fixes
- Shared-world runs finish with plain participant captions and app-use wording in the CLI, and
the Observer uses those captions for recording titles. Older terminal and scripted runs show
participant names in the Observer and the TUI, including dry runs without actor traces (row 14).
(#1655)
Known gaps
- Nothing Mac-specific in #1658 ran on a Mac: the
limactl list --jsonfield names,limactl edit --cpus --memoryon a stoppedvzinstance, the new VM size, and the capacity refusal on a
real 8 GiB VM. - The out-of-memory path was not observed for real. It is unknown whether Docker records
oom
when the VM-wide OOM killer kills a desktop that is still under its container limit, and a
Chromium crash inside the guest's own 2 GiB is not detected. - The Codex participant's impressions ran only through its captured final-account shape, and a
Claude local-agent participant records none. - The Observer's "What they said at the end" ran in headless Chrome on three live runs in the
release smoke; no other browser opened it.humanish review,review.mdand the TUI do not
show impressions. - The impressions ask describes its kinds in visual terms ("looked confusing or hard to read",
"looked broken, unpolished or out of place"), which can steer a participant toward visual
critique whatever its persona. (#1654) - When the OpenAI organization refuses server-side conversation state, humanish carries the
conversation itself and cuts it to half of a 64,000-token estimate once it passes that budget,
turning the oldest turns into a text note. The impressions request after a long session then
reads the cut conversation: in the smoke, a 30-turn participant's last turn carried 53,693 input
tokens and its impressions request 25,750. The trace's conversation record lists participant
requests only, so it does not say what the impressions request carried. - A project made by an earlier release keeps starter personas without a background, and
init
does not update them, sostudy check try-livethere warns until the persona file is edited. - Automatic analysis refuses before any request when its estimate passes $3: the smoke's
two-participant fan-out (30 and 27 turns) estimated $3.11 and neededanalyze --max-cost 4.
The benchmark's $1.75 per-analysis cap refused one planted analysis estimated at $1.81.
Impressions add up to six evidence items per participant; how much of an estimate they account
for was not measured. - Design findings cannot say that text or a click target is too small by measure. (#1638)
- A reviewer cannot add a note at a moment in a recorded run. (#1635)
- The shared secret scrubber leaves
[REDACTED_...]-shaped spans alone, reads percent-encoded
UTF-8 one byte per character, and has a quadratic marker check. (#1646) humanish verifyand bundle export read percent-encoded UTF-8 one byte per character. (#1645)
Package
humanish 0.113.0 packs to 2.4 MB, 7.6 MB unpacked, 932 files (npm pack on release/0.113.0 at
c47238c). Against 0.112.0 it adds dist/actors/computer-use/loop/impressions,
dist/cli/update-check, dist/cli/update-check-worker, dist/keys/key-status and
dist/substrates/local/capacity (.js and .d.ts each) and drops nothing. The package exports
only ., with the same 13 values and 31 types as 0.112.0 (pnpm api:proof). It needs Node
22.19 or newer, as before, and @e2b/desktop 2.3.2 or newer as an optional peer.
The release tarball's shasum is 95fe84fbb20d36f58ec824359e9fe43c9e2c3f2d and its integrity
sha512-l9H7jPUTBRgfyHZctWaOdEEBJDRUJfUGS523lzrpXWO4BatHo/CvVgHpWF0239pzblo5OOV11mHeEERZZjEGTA==.
The registry serves the same values: npm published 0.113.0 on 2026-10-07 from the tagged commit, with a signed provenance statement, and the post-publish check passed 93 of 93 against the registry and unpkg.
Checked before release
A live smoke on a tarball packed from main 323e65a, in fresh projects under /tmp with a fresh
HOME and npm cache, DO_NOT_TRACK=1, and @e2b/desktop 2.4.0 beside humanish. Keys came from
the user key store through key discovery, with XDG_CONFIG_HOME pointing at it; no --dotenv
was passed. The store's OpenAI organization refuses server-side conversation state, so every
participant ran with the conversation carried by humanish (explicit_context). Dry runs,
refusals and the same CLI commands also ran on npm 0.112.0. Priced spend: $5.10 for the three
live runs, $3.02 of it in analysis, then $5.83 for the benchmark, $0.87 for one of its analyses
run by hand, and $0.25 for the dogfood run.
- try-live, as
initwrites it: reached the goal in 10 turns, input growing every turn from
2,888 to 18,261 tokens, then 18,645 for the impressions request; $0.85. The participant gave 5
impressions (unclear 2, unfinished 1, liked 2), among them "The second table appeared
overlapping the first on the canvas, which made the initial layout look unpolished." The
analysis, atstudy-evidence-8, cites an impression in each of its 3 findings and in 1 of its
3 design findings.reviewprinted headlines and design findings as 0.112.0 did. The Observer,
served byobserve --run <id> --servein headless Chrome at 1440x900 and 390x844, showed "What
they said at the end" in the participant's Feedback tab with the groups "Confusing or hard to
read", "Looked broken or unfinished" and "Worked well", each impression with a link to its
recorded message, and no page errors. - A computer-use fan-out, 2 participants on drawDB: both reached the goal, input growing every
participant turn, 2,948 to 53,693 over 30 turns and 2,982 to 48,773 over 27; the impressions
requests carried 25,750 and 49,423 (Known gaps). 6 impressions each, among them "I saw “Record
version” fail with only a “Failed to record version” message. With no saved version or recovery
guidance, I would hesitate to trust version history with important work." Automatic analysis
was refused at its $3 default ("its estimate ($3.11) is over the default $3 cap") and ran after
withanalyze --max-cost 4: 7 findings, 6 cite an impression. $3.48. - shared world, 2 participants, live: both reached the goal, input 2,760, 4,430, 6,154 and 2,766,
4,436, 6,165, then the impressions requests; 4 and 5 impressions, including both
unlike_my_workkinds, which the analysis's first design finding cites; $0.77. The closing
output reads as row 14. Its stdout, stderr,review, captions, summaries and persona name hold
none of swarm, plane, coherently, "overlap proven",goal_satisfied,type:,surface:or
case:. Of the 663 strings inobserver-data.json, 10 hold one, all machine values #1655 keeps:
the event idevent-001-plane, the event typesconcurrent-shared-world.plane.provenanceand
concurrent-shared-world.session.goal_satisfied, and the completion reasongoal_satisfied.
The Observer's visible text holds none. - Every participant in the three runs ended the session itself, and each impressions request
completed (debrief.triggerparticipant_end,statuscompleted).counts.messagesstayed 1
for each. study checkon the four starter studies: no warning on 0.113.0, which writes personas with a
background. On a study whose mission is1. Click "New task". 2. Type "Buy milk" into the title field, then press Save. ...: exit 0, "warning: actor.mission reads like a script, which can hide
where a participant would get lost. Matched lines:" and the line quoted. On a persona without a
background: exit 0, "warning: Participant lane-01 has no persona background because persona
no-bg-user has no readable background. ..." npm 0.112.0 printed neither. 0.113.0 on a project
initmade with 0.112.0 warns ontry-live.runtime statuson this machine (62.7 GiB, 16 CPUs): "... so 20 desktops fit at once. The
desktops share 16 CPUs, so they run slower when all of them are busy.";--jsoncapacity
desktops20.doctor --study local-browseradded "local desktop capacity: ... This study runs
1 at once, so it fits."runtime setup --memory 8 --cpus 4exited 2 naming the Mac.humanish keys: the four keys with their sources (~/.config/humanish/keys.envand
gh auth token), "Also in the user store: ANTHROPIC_API_KEY. humanish does not use it today.",
and no value.keys setwithout a terminal: exit 2 namingkeys set e2b --stdin.
humanish --helpandkeys --helplist nohelp.- The update notice, from a copy of the main build with its version set to 0.111.0 and a cache
naming the registry's latest, 0.112.0, checked that minute: one line at a terminal, "humanish
0.111.0 is out of date; the latest is 0.112.0. Run npx humanish@latest to use it.", and none on
the next command, withDO_NOT_TRACK=1,CI=1,--json, or without a terminal. Telemetry was
off for it throughtelemetry.json, since the notice needsDO_NOT_TRACKunset.doctor
showed the cached version. - After every run,
reclaim --checkexited 0 withclean, andverifyexited 0 with
local_only. Raw sandbox ids appear only insandbox-receipts.ndjson, and no key value from
the store appears in any run or kit file, as written, base64 or hex.
On the release commit c47238c:
pnpm release:checkexit 0 in 4.0 min: 466 test files passed and 10 skipped, 7,479 tests
passed and 12 skipped; TUI 158 tests; lint 440 warnings at its cap of 440; api:proof "13 values
and 31 types match"; public-surface scan of 2,903 text files and 523 binary assets;npm pack --dry-runshasum as above.pnpm docs:generatewrote 0.113.0 into cli.mdx. It passed again
on the benchmark commit a287f76, in 3.5 min with the same test and lint counts, a
public-surface scan of 2,905 text files, and the samenpm pack --dry-runshasum. The first run
on that commit failedprose:check(prose.evidence.issue-refs103, cap 102) on an issue
number in the benchmark README; the commit was amended to drop it.pnpm release:dogfood(the script run with Node, keys read from the user store by Node's
--env-fileinto its environment, and a temporary HOME): "release:dogfood ok", verdict
blocked(blocked_approval), no-spend satisfied, $0.25. The participant raninitand the
free dry run on the installed 0.113.0 and reported "Passed all 16 verification checks; evidence
wasshare_ready." and "It recommendshumanish run try-live." It stopped at the credentials
the gate withholds.verifyon the run:share_ready, 16 of 16;reclaim --checkclean.- The post-publish check on the release tarball: 87 of 87 passed. Its 5 new rows cover
keys
status and thekeys setrefusal,helphidden from Commands, the update notice at a
pseudo-terminal (#1660), thestudy checkwarnings with the starter studies warning on nothing
(#1656), andruntime status --jsoncapacity with the Linux refusal of--memoryand the
doctor capacity row (#1658). The same script on npm 0.112.0 passed 82 and failed exactly those
5 rows. - The consumer check: the 0.112.0 check's library and CLI consumer, plus four library steps and
seven CLI lines for this release. Typecheck 0 errors on both versions. Every output line is the
same on npm 0.112.0 and the release tarball except the new steps and lines and the 0.111.1
egressAllowdry run, which records one morestudy.warning(the consumer project has no
persona files, so the no-background warning joins it). pnpm benchon c47238c with its defaults (3 runs per arm,openai-computer-use,neutral
mission), committed in a287f76: report recall 10/15, analysis recall 6/10, 0 invented on the
clean arm, $5.83 estimated. 0.112.0 scored 11/15 and 9/15. The second planted analysis was
refused at the benchmark's $1.75 per-analysis cap (estimate $1.81), so analysis recall has 10
chances; run by hand ($0.87), that analysis lists D1, D3, D4 and D5. A line the rubric left
unresolved in the first planted report and one in the third planted analysis are D5, which puts
both recalls at 11/15 read by hand; 0.112.0 read 11/15 and 10/15 by hand. The planted-arm
invented line, "the edit save button did nothing." (X4), is D5. No planted participant typed
more than 30 characters in one action, so none met D2. The benchmark's persona is the repo's
synthetic-new-user, which #1656 rewrote to lead with a background, so its participants' prompt
changed; at three runs per arm (report recall interval 0.42 to 0.85) the numbers cannot show a
persona effect. One clean participant said it did not reach the goal after a refresh emptied the
list, which both builds do, and that it would not keep using Taskly; all six of 0.112.0's
participants reported reaching it. Every participant's input grew each turn, and each gave 5 or
6 impressions. E2B listed 0 tagged sandboxes after it.
Not verified
- macOS and Windows. Every check ran on Linux x64 with Node 24.12.
- A live run on the published package: the live smoke ran on a main tarball, and the post-publish
rows are offline dry runs, refusals and a local scripted run. - The impressions request with server-side conversation state (
threaded): the store's OpenAI
organization refuses it, so every live participant here carried its own conversation. - The
keys setwalk at a terminal, with its hidden prompts: not run here. - The update check's own registry request: the cache was written by hand with a fresh
checkedAt,
so no worker started. doctor'snext:line: this machine's local runtime is installed. The Linux capacity warning
needs a study with more than 20 desktops at once here and was not run.- A feedback draft quoting an impression, the TUI footer, and older terminal and scripted runs
re-captioned in the Observer and the TUI: their tests inrelease:checkcover them, and no
check here ran them. - The local VM, email capture and receiving, local-agent participants, terminal and scripted
routes: not run live on 0.113.0.