Repository navigation
0.120.0: Analysis completes under Bun, runs past 16 participants print a bounded list, Observer counts participants by status
humanish 0.120.0 completes an analysis run from a Bun process, prints a run of more than 16
participants in a bounded number of lines, and counts and filters the Observer's participants by
status and persona. Under Bun, undici is Bun's built-in module, whose Agent has no destroy(),
and the analysis provider's teardown after each request threw a TypeError that replaced the
request's result, so every automatic analysis from a Bun process ended unknown with
AUTOMATIC_ANALYSIS_OUTCOME_UNKNOWN and kept no receipt. A failing teardown is now ignored.
humanish watch and humanish run print a computer-use run of more than 16 participants as one
line counting them by status, up to 16 participants that did not pass, and a not listed: line.
Each participant line holds its closing message on one line cut at 160 characters, and a warning
printed the same for each participant prints once. The Observer's participant grid counts the
participants by status when they ended more than one way, and pressing a count filters the grid.
Its view options gain a Persona filter, a filter change returns to the first page, and Next page
and Previous page open the new page at its first participant.
This is a minor release because watch and run print a computer-use run of more than 16
participants in a new shape and cut each participant's closing message at 160 characters, which a
script reading their terminal output would notice. Their --json output is unchanged. The package
exports the same 13 values and 31 types as 0.119.0 and has the same file list, dependencies and
engines. It removes or renames no command, flag, study field or bundle field, and verify,
review and runs print the same for the same bundle. These outputs change for the same input:
| # | 0.119.0 | 0.120.0 | What to do |
|---|---|---|---|
| 1 | watch and run printed one line per participant for a computer-use run of two or more participants. In the smoke, 20 participants (2 passed, 18 gave up) gave 144 stdout lines, 47 of them the participant list; in the consumer check, a dry run of 40 gave 99 stdout lines with 40 participant lines. |
Above 16 participants: "participants: 20 · 2 passed · 18 abandoned", the first 16 participants that did not pass, one line each, and "not listed: 4 participants (2 did not pass). The Observer shows each one." The smoke's run gave 96 stdout lines, 18 of them the list; the dry run of 40 gave 61, with "participants: 40 · 40 dry run, nothing ran live" and "not listed: 40 participants. The Observer shows each one." A run of 2 to 16 participants still lists each one. (#1744) | Read each participant from --json (lanes[]), run.json or review. Match participants: and not listed: for the counts. |
| 2 | Each participant line of a run of 2 or more participants ended with the participant's whole closing message, line breaks included: 2 or 3 terminal lines per participant in the smoke. | The closing message is on the participant's one line, whitespace collapsed, and cut at 160 characters with …: "participant buyer-01: abandoned (gave_up) · participant outcome · DID NOT REACH THE GOAL. The page only displays bakery information and today's loaf ... or oth…". A run of one participant prints its session: line whole, as before. (#1744) |
Read the whole message from --json (lanes[].session.reason), run.json or review. |
| 3 | watch and run printed every warning the result carried, so a warning added once per participant printed once per participant: 20 copies of "Screenshots are unblurred for local use ..." for 20 participants, 86 warning lines in all. |
Each distinct warning text prints once, on all four routes: 1 copy, 67 warning lines. Warnings whose text names a participant (window clipping and fill, slow DevTools, persona background) still print once per participant. --json keeps every copy. (#1744) |
Count warnings from --json's warnings. |
| 4 | An automatic analysis run from a Bun process (runStudy under Bun 1.3.11) ended state: unknown with AUTOMATIC_ANALYSIS_OUTCOME_UNKNOWN, analysisId: null and error.code: ANALYSIS_UNAVAILABLE, kept only start.json in analysis-attempts/<id>/, and recorded no cost, although its request was sent: the job ran 80 s. The analysis provider threw "TypeError: dispatcher.destroy is not a function" on a 200 and on an HTTP 500. |
It ends complete with analysis/<id>/analysis.json and receipt.json, as under Node: in the smoke, one participant's analysis was expected at $0.80 and billed $0.20 under Bun ($0.25 under Node). A provider failure under Bun is recorded as that failure (provider_http_error for an HTTP 500, with a fake fetch). (#1753, fixes #1752) |
Nothing. To analyse a run that 0.119.0 left unknown under Bun, run humanish analyze --run <id>; its dry run on 0.120.0 admits such a run at an expected $0.80 (one participant). Spend from those 0.119.0 requests is in no receipt. |
Added
- The Observer's participant grid counts the participants by status in a row under the run's
tally, when they ended more than one way. Pressing a count shows only those participants, and
pressing it again shows everyone. The view options gain a Persona filter when the participants
have more than one persona. In the smoke, 20 participants showed "Reported complete 2" and "Gave
up 18"; pressing "Gave up" showed 18 cards. (#1744, from #1737)
Changed
humanish watchandhumanish runprint a computer-use run of more than 16 participants in a
fixed number of lines: one line counting the participants by status, up to 16 participants that
did not pass, and how many it did not list. A run of 16 or fewer still lists every participant.
Each participant's line holds its closing message on one line, cut at 160 characters, where it
used to print the whole message. A warning repeated for each participant prints once (rows 1 to
3). (#1744, from #1737)- In the Observer, a changed grid filter shows the first page of participants. (#1744, from #1737)
Fixes
- An analysis run from a Bun process completes. Under Bun,
undiciis Bun's built-in module,
whoseAgenthas nodestroy(), and the provider's teardown after each request threw a
TypeErrorthat replaced the request's result. Every automatic analysis then endedunknown
withAUTOMATIC_ANALYSIS_OUTCOME_UNKNOWN, even when the model answered, and kept no receipt for
requests already sent. A failing teardown is now ignored (row 4). (#1753, fixes #1752) - In the Observer, Next page and Previous page under the participant grid now open the new page at
its first participant. The grid kept its scroll position, so turning the page from the bottom of
36 participants showed the end of the next page, about 10,000 px past its first participant on a
phone. (#1744, from #1737)
Known gaps
- An error raised after an analysis request is dispatched, other than the teardown, is still
recorded asunknownwith no receipt, so the spend of requests already sent is not recorded.
#1752 suggested recording it asfailedwith the usage known; #1753 fixes only the teardown.
(#1752) - No live run of 40 or 100 participants has been made. The smoke ran 20 on one Observer page;
#1744 measured the grid, paging andwatchat 40 and 100 with fixtures built from one recorded
participant. (#1737, #1744) - Above 16 participants,
runstill prints the per-participant warnings whose text names the
participant: in the smoke, 20 participants gave 67 warning lines, 20 of them persona-background
warnings and 40 window clipping and fill warnings. The plan and progress on stderr print one or
more lines per participant (191 stderr lines for 20). The shared-world formatter still lists every
participant on its own line. (#1744, #1739) watchprints the run's block and then the Observer's block ("humanish observer rendered"), and
both carry the run-level warnings, so a dry run prints "This is a dry run: ..." twice. 0.119.0
does the same.- Two of the Observer browser proofs wait without a time limit, so a stall hangs the CI step until
its 5-minute timeout without naming the case or step. (#1750) - The Observer's
spend.requestscounts an analysis attempt as one request. A cohort analysis of
more than 16 participants sends one request per cohort and a merge request, and its
study-analysis.jsonreads"requests": 1. (#1743) - On a
local-treeorclonesubject, the first participant holds the others at the pipeline gate
until its desktop is ready. Participants due before the gate opens start when it opens, late, and
the plan's schedule does not count the gate. A failed first desktop skips every other
participant. (#1741) - A participant waits for its scheduled start before it is skipped for the budget, so a run whose
budget is crossed early idles until its last start, with no sandbox.runprints such a
participant as "blocked · unknown ending" and leaves out the reason, which run.json holds.
(#1745) - In-progress bundles and the Observer show a participant waiting for its start as running. The
plan's peak and the provisioned shared world's app deadline count each session at its full budget
and leave out each desktop's start-up time; with several waves on a provisioned shared world,
that start-up comes out of the 10-minute teardown buffer. (#1745) doctor --studyof a study of more than 16 participants prints "ok post-run analysis" and does
not say that the default $3 cap refuses its analysis;runandstudy checksay so. (#1742)- A merged report can name the cohorts: the 0.119.0 smoke's 24-participant summary opens "Across
both cohorts". The merge request'sAnalysis requestingline reads "0 evidence items, 0
captures." (#1742) rundoes not print the warning that namesHUMANISH_E2B_MAX_CONCURRENT_SANDBOXESwhen the
limit holds participants to waves; the run records it as astudy.warningevent, and the
terminal shows only the plan line. (#1739)runprints its analysis line from the declared participant count, so a--countoverride does
not change it, and a--countabove 64 with real email receiving still fails at run start.
(#1739)- A held Enter in
humanish tuican still confirm an armed action on its first key repeat. The
400 ms floor ignores repeats that come sooner, and a system's initial repeat delay is often
longer (500 ms by default in GNOME). (#1730) - A
serve.startthat bash cannot parse still waits outreadyTimeoutMs. The scripted and
shared-world routes give a failed build their generic failure code; the message carries bash's
error. (#1734) - On a desktop with speech, heard speech reaches the participant only with a screenshot and the
guest holds 8 utterances between two screenshots, so a long wait in a talking call can overflow
it; a wait there stays 30 seconds unless the study raisesactor.maxWaitMs. (#1724) - A host that sleeps during a hosted run is reported as provider stalls and failed participants.
(#1740) - Two contained-file races from #1669 remain. They need a second local process, running as the
same user, to swap a directory in the run between two filesystem calls: temporary-file cleanup
still unlinks by path (item 1), and the reviewer-notes listing still listsnotes/by path and
can follow a swapped directory (item 3). Storage validation and the scan still list each folder
whole before the 10,000-entry cap applies, verify has no total read budget, and a legitimate
sandbox-receipts.ndjsonover 32 MiB is refused. (#1669) - Coding agents still do not tell their person about
humanish tui. (#1701) - Path redaction in terminal transcripts still removes the text after a path up to the next space,
such as a line break and the next header aftercwd:. Nothing leaks; text a reviewer needs is
lost. (#1702) - A hosted computer-use participant cannot type keysyms outside US printable ASCII (
é,€) as
keys; it has to type them as text. (#1713) - The routes still take
E2B_DOMAINfromprocess.envon create and ignore one in
RunStudyOptions.env. (#1717) - From #1646, after #1684: a 4-byte value inside a longer base64 run is not found, two further
decodings at a marker edge are not read,RunSecrets.spans(the terminal route's per-chunk
replacement) has no final check, and a value that is part of a marker nests markers. (#1646) - While a live run works, its study screen in
humanish tuireads "1 run, none live". - Design findings cannot say that text or a click target is too small by measure. (#1638)
- The impressions ask describes its kinds in visual terms, which can steer a participant toward
visual critique whatever its persona. (#1654) - Nothing Mac-specific has run on a Mac: the Lima VM sizing and
runtime setup --memory --cpus.
Package
humanish 0.120.0 packs to 2.5 MB, 7.9 MB unpacked, 972 files (npm pack on release/0.120.0 at
1122987, shasum ee45f9f5e398f9b9327ff3deecfeb09f117b724e), the same file list as 0.119.0. The
package exports only ., with the same 13 values and 31 types as 0.119.0 (pnpm api:proof). Its
dependencies, optional peer and engines are the same as 0.119.0's: Node 22.19 or newer, and
@e2b/desktop 2.3.2 or newer as an optional peer.
Checked before release
A live smoke on a tarball packed from main cf75cc6, in fresh projects under /tmp with a fresh HOME
and npm cache, DO_NOT_TRACK=1, and @e2b/desktop 2.4.0 beside humanish. CLI commands got keys
from the user key store through key discovery, with XDG_CONFIG_HOME pointing at it. The
runStudy scripts, which do no key discovery, got the same store through the runtime's own
--env-file (Bun 1.3.11 and Node 24.12.0) into that one process. No --dotenv was passed. The
store's OpenAI organization is non-ZDR, so every OpenAI participant ran threaded; its E2B key is
on a Pro plan. Controls ran the same study files on npm 0.119.0 before #1753 merged. Priced spend:
$2.62 for the smoke ($1.13 of it in three analyses; $0.60 in the npm 0.119.0 controls, whose Bun
analysis request was billed and not recorded), then $0.26 for the dogfood run and $5.87 for the
benchmark.
init --yeswrites the same files as npm 0.119.0, anddoctor(the concurrent-sandboxes row
unset, at 100 and atlots),doctor --study try-live,keys,study check try-liveand
analyze --helpprint the same apart from the cwd.runStudyfrom a Bun process (row 4): one participant on a static page with a declared analysis.
On main under Bun 1.3.11 the participant endedgoal_satisfiedand the automatic analysis ended
complete, withanalysis.jsonandreceipt.jsonwritten (expected $0.80, billed $0.20; 9
cited evidence ids, all resolved). The same script under Node 24.12.0:complete(expected
$0.80, billed $0.25). On npm 0.119.0 under Bun:unknown,AUTOMATIC_ANALYSIS_OUTCOME_UNKNOWN,
onlystart.json, no cost recorded, after an 80-second job.- 20 participants with
HUMANISH_E2B_MAX_CONCURRENT_SANDBOXES=100and 2-minute sessions (rows 1
to 3): 2 readers passed and 18 participants asked to order on a page without an order form gave
up, on both versions; E2B listed the run's 20 sandboxes at once.runprinted 96 stdout lines
against npm 0.119.0's 144: the list was 18 lines against 47, the longest closing message 160
characters, and the screenshots warning 1 copy against 20. The default analysis was refused
before it started at the $3 cap and recordedskippedwithAUTOMATIC_ANALYSIS_ADMISSION_REFUSED
on both. In headless Chrome the candidate's Observer showed the status row "Reported complete 2",
"Gave up 18"; pressing "Gave up" showed 18 cards and pressing it again 20; the Persona select
listed both personas. npm 0.119.0's Observer of its own run had neither. - try-live, as
initwrites it: reached the goal in 10 participant turns, input growing every
turn from 2,888 to 18,100 tokens, then 18,463 for the impressions request; cached input on each
turn from the second was the previous turn's input minus 3;threadedon all 11 requests. $0.25
run. Its automatic analysis, one request, was admitted at an expected $1.03 (admittedCostUsd
$1.13, worst case $2.01, cap $3) and billed $0.67, with 2 findings and 4 design findings whose 29
cited evidence ids all resolve. It gave 5 impressions (unclear, unclear, unfinished, liked,
liked). verify,reviewandruns(text and--json) on copies of the six live runs: all 38 outputs
byte-identical on main and npm 0.119.0. A dry run ofanalyze --runon npm 0.119.0'sunknown
Bun run is admitted on main at an expected $0.80.- After every live run,
reclaim --checkexited 0 withclean, andverifyexited 0 with
local_only. Raw sandbox ids appear only insandbox-receipts.ndjson. E2B listed 0 tagged
sandboxes before each batch, and after the smoke 1 tagged sandbox from none of its runs (the
dogfood ran then), and 0 after the benchmark. No key value from the store appears in any run
or kit file, as written, base64 or hex (11,386 files).
On the release branch:
pnpm release:checkon the release commit 1122987: exit 0 in 4.1 min, 515 test files passed
and 10 skipped, 8,042 tests passed and 12 skipped; TUI 192 tests; lint 426 warnings at its cap of
426; api:proof "13 values and 31 types match"; public-surface scan of 3,068 text files and 527
binary assets;npm pack --dry-run972 files with shasum
ee45f9f5e398f9b9327ff3deecfeb09f117b724e.pnpm docs:generatethere wrote 0.120.0 into cli.mdx.
On the benchmark commit bd5a7d2, the release head: exit 0 in 3.9 min, the same test, lint and
api:proof counts, a public-surface scan of 3,070 text files and 527 binary assets, and the same
npm pack --dry-runshasum, so the tarball the checks below used is the head's package.pnpm release:dogfoodon 1122987 (the script run with Node, keys read from the user store by
Node's--env-fileinto its environment, and a temporary HOME): "release:dogfood ok", verdict
passed(goal_satisfied), no-spend satisfied, $0.26. The participant raninit, the free dry
run,verify(16 of 16,share_ready) andreviewon the installed 0.120.0, and did not start a
paid live run.verifyon a copy of the run:share_ready, 16 of 16;reclaim --checkclean.
0.119.0's dogfood endedblocked(blocked_approval) after the same steps.- The post-publish check on the release tarball from 1122987 (
TMPDIR=/tmp,HUMANISH_SPECthe
tarball): 125 of 125 passed. Its 2 new rows coverwatch --dry-runof 16 and 40 participants
(#1744) and the analysis provider under Bun and Node with a fake fetch (#1753). The same script on
npm 0.119.0 passed 123 and failed the 2 new rows. A first pass failed row 117 on both versions
(124 and 123 of 125): it counted a warning repeated anywhere inwatch's stdout, andwatch
prints the run's block and then the Observer's, each with the same two run-level warnings. The
row now checks the run's block, where 0.120.0 dedupes; both passes are inpost-publish/. - The consumer check: the 0.119.0 check's library and CLI consumer, plus 4 CLI lines for this
release, on the release tarball from 1122987. On that tarball the consumer typechecks with 0
errors, as on npm 0.119.0, and needs no migration. The library steps are the same on both
versions, and of the 138 CLI lines from the 0.119.0 check, 136 are the same; the two that differ
are the dry runs of 17 and 24 participants, which now print "participants: 17 · 17 dry run,
nothing ran live" (row 1).watch --dry-runof 16 participants prints the same 51 lines; of 40,
61 lines against 99 (row 1). The package's analysis provider with a fake fetch under Bun returns
completedfor a 200 andprovider_http_errorfor a 500, where npm 0.119.0 throws the
TypeErroron both; under Node both versions return the same (row 4). pnpm benchon 1122987 with its defaults (3 runs per arm,openai-computer-use,neutral
mission), committed in bd5a7d2: report recall 12/15, analysis recall 11/15, 0 invented on either
arm, $5.87 estimated. 0.119.0 scored 10/15 and 10/15 at $6.12 (11/15 each read by hand). The bench
skipped the clean 3 analysis asskipped_budget, since spend stood at $5.87 and that analysis
could have passed the $7 cap, so the analysis column covers 3 planted and 2 clean participants.
D1 and D5 account for the rise: all three planted reports name each, where two did on 0.119.0.
Read from the traces' click coordinates as #1749 does, every planted participant committed its
first edit with Save (3 of 3 on both versions) and pressed Clear completed after checking a task
(3 of 3, against 2 of 3 on 0.119.0). The D1 rise follows that reach; D5 was reached 3 of 3 on both
and named 3 of 3 against 2 of 3. Planted 2's unresolved analysis finding "Edited task text
reverted after Save and filter changes" describes D5, so analysis recall read by hand is 12/15.
Three runs per arm cannot separate this rise from the noise #1749 measured (52/80 planted-report
recall each for npm 0.114.0 and main d183d1a in one hour). The analysis cap was the dry run's
admittedCostUsd, $1.21, against bills of $0.64 to $0.93. No planted participant typed more than
28 characters in one action, so none met D2. Every participant ran threaded, ended on its own and
gave 6 impressions.
Not verified
- macOS and Windows. Every check ran on Linux x64 with Node 24.12 and Bun 1.3.11.
- A live run on the published package: the live smoke ran on a main tarball, and the post-publish
rows are offline dry runs, refusals, local runs and a fake fetch. - Bun versions other than 1.3.11 (#1752 saw 1.3.13), the
humanishCLI run under Bun, and a Bun
analysis of more than 16 participants: the smoke's Bun analysis sent one request, where the
adopter's 34-participant run sent cohort requests and a merge. The fix is in the per-request
teardown that every cohort and merge request runs. - A live analysis request that fails under Bun; the fake fetch covers an HTTP 500.
- A live run of 40 or 100 participants, live paging in the Observer, and the shared-world,
terminal and scripted routes' warning dedupe on a live run. --jsonofwatchandruncompared between versions on a live run; #1744 says it is unchanged.- What the npm 0.119.0 control's Bun analysis request cost: 0.119.0 recorded no receipt.
analyze --runsent live on a run that 0.119.0 leftunknown; only its dry run was tried.