Repository navigation
0.118.0: Up to 100 participants, E2B concurrent sandboxes, study check plans like run, analysis range
humanish 0.118.0 lets a computer-use study have up to 100 participants, up from 16, holds a
shared-world study to the same 100, and runs them within the number of sandboxes the E2B plan runs
at once. The new setting HUMANISH_E2B_MAX_CONCURRENT_SANDBOXES gives that number (20 when unset,
E2B Hobby's limit; Pro runs 100). A study with more participants than that runs them in waves, an
execution.concurrency above it is refused before any sandbox is created, and doctor shows it. A
live run of more than 16 participants records its automatic analysis as skipped with
AUTOMATIC_ANALYSIS_PARTICIPANT_LIMIT, because one analysis reads at most 16 participants. study check now plans a study the way humanish run does and fails a study the run refuses, and a
shared-world or scripted clone study whose sandbox deadline passes the ceiling is refused at plan
time. The analysis line that run, study check and the TUI print ends its expected range at the
largest run admission admits ($0.75 to $2.83 for one participant at the default $3 cap, where it
read $3.79) and states the rule as admission applies it. An in-process local-app run records
subject.source: local-app, a desktop-cli dry run names its product, and verify names an unlisted
folder and the 10,000-entry stop in their own sentences. In humanish tui, the key legend says what
Enter and Esc do while an action is armed, a study's median cost includes its analyses, and Esc
after entering a key returns to the screen the keys were opened from.
This is a minor release because it raises the participant limit, adds a setting that bounds how
many participants run at once, and changes outputs for the same input. A study of 17 to 100
participants, which 0.117.0 refused, now runs, and its automatic analysis is skipped. An
execution.concurrency above HUMANISH_E2B_MAX_CONCURRENT_SANDBOXES is refused. study check now
exits 2 for a study run refuses at plan time, where it passed. A shared-world or scripted clone
study whose sandbox deadline passes the ceiling is refused at plan time, the scripted one with the
new code HUMANISH_SCRIPTED_SUBJECT_INVALID. An in-process local-app run records
subject.source: local-app, which npm 0.117.0 cannot read. The analysis line, doctor's post-run
analysis row and analyze --help state the admission rule in new words, the line's range ends lower,
and study check --json can carry analysis.refusedFromUsd in place of
analysis.expectedCostUsd. A parsed shared-world study no longer carries an
execution.concurrency it did not declare, and verify and the desktop-cli dry run reword
sentences. The package exports the same 13 values and 31 types as 0.117.0; the exported RunBundle
type's subject.source gains local-app. It removes or renames no command, flag, study field or
bundle field. These outputs change for the same input, which a script could notice:
| # | 0.117.0 | 0.118.0 | What to do |
|---|---|---|---|
| 1 | A computer-use study of more than 16 participants, and a shared-world study of more than 16 members, was refused when read: "A computer-use study runs at most 16 participants, and this one declares 17. Each participant is a paid desktop and they all run at once, so no setting raises the cap; split the roster across studies.", HUMANISH_STUDY_INVALID, exit 2. A participants group with count: 1000000000 ran the parser out of memory: exit 134 after 44 s in the smoke. |
Up to 100 plan and run. More is refused: "A study runs at most 100 participants, and this one has 101. One run keeps every participant's evidence in one bundle and one Observer view, and humanish supports runs of up to 100; split the participants across studies.", HUMANISH_STUDY_INVALID when read, HUMANISH_COMPUTER_USE_FANOUT_INVALID for a --count above 100. count: 1000000000 is refused in under a second. A study with real email receiving and more than 64 participants is refused when read. (#1739) |
Match the refusal by its code. A study of more than 16 participants needs 0.118.0 to run, and npm 0.117.0's TUI lists it by its id with "no description in the manifest". |
| 2 | Every participant of a hosted study ran at once, at most 16. | At most HUMANISH_E2B_MAX_CONCURRENT_SANDBOXES desktops run at once (20 unset), counting a provisioned shared world's app sandbox; later participants start as earlier desktops end. In the smoke, 24 participants peaked at 20 sandboxes unset and at 24 with the setting at 100. The run records a study.warning event naming the setting; run prints its plan line ("concurrency 20, 2 waves"). An execution.concurrency above the limit is refused before any sandbox: "execution.concurrency runs 24 participants at once, and humanish runs at most 20 sandboxes at once, E2B Hobby's limit, when HUMANISH_E2B_MAX_CONCURRENT_SANDBOXES is unset. Lower execution.concurrency to 20 or less, or remove it to run 20 at a time. If your E2B plan runs more (Pro runs 100), set HUMANISH_E2B_MAX_CONCURRENT_SANDBOXES to 24 or more.", HUMANISH_COMPUTER_USE_FANOUT_INVALID. A value of the setting that is not a whole number of 1 or more refuses every hosted plan. (#1739) |
On a plan that runs more than 20 sandboxes, set the setting to its limit (100 on Pro). A study 0.117.0 accepted runs as before: 16 participants, or 16 members and an app sandbox, fit under 20. |
| 3 | doctor had no concurrency row. |
doctor prints "e2b concurrent sandboxes: 20 at once, E2B's Hobby limit. ..." unset and "100 at once, from HUMANISH_E2B_MAX_CONCURRENT_SANDBOXES. ..." at 100. A value it cannot read fails the row (- missing e2b concurrent sandboxes: HUMANISH_E2B_MAX_CONCURRENT_SANDBOXES must be a whole number of 1 or more (got "lots"). ...) and doctor exits 2. (#1739) |
A script that counts doctor's rows sees one more. |
| 4 | No run had more than 16 participants. | A live run of more than 16 participants prints "analysis: not run, because the study has more than 16 participants and automatic analysis reads at most 16", and its JSON result records the analysis as skipped with AUTOMATIC_ANALYSIS_PARTICIPANT_LIMIT. It exits 0 when the study declares no review.analysis (both 24-participant smoke runs) and 2 when it declares one. run, study check, doctor --study and the TUI say so before the run: "After live runs: automatic analysis will not run. This study has 24 participants, and automatic analysis reads at most 16. ..." (#1739) |
Set review.analysis: false on such a study, or expect exit 2 when it declares review.analysis. |
| 5 | study check passed a study run refuses at plan time, such as a 90-minute sandbox deadline at the 60-minute ceiling, a local-app study or a mode: live preview study: exit 0, "humanish study check passed". |
It plans the study as run does and exits 2 with a - fail plan: row carrying the run's message and code, such as HUMANISH_COMPUTER_USE_SUBJECT_INVALID, before any reachability probe. A study that plans prints the same as before. (#1733) |
Read a failed plan row as the run's refusal. |
| 6 | A shared-world study whose participant sandbox deadline passes HUMANISH_E2B_MAX_SANDBOX_MINUTES, or a scripted clone study whose subject sandbox does, planned, and its dry run finished; a live run reached E2B, which refuses a sandbox past the plan's lifetime. |
Refused at plan time, dry runs too: "execution.timeoutMs 55m derives a 65m deadline for each participant's sandbox, ... set HUMANISH_E2B_MAX_SANDBOX_MINUTES to 65 or more.", HUMANISH_SHARED_WORLD_INVALID; on the scripted route "... derives a 70m deadline for the subject sandbox ...", with the new code HUMANISH_SCRIPTED_SUBJECT_INVALID, which also refuses a value of the setting it cannot read. (#1733) |
Match the new code. Lower execution.timeoutMs or raise the setting to your plan's limit. |
| 7 | The analysis line read "expected $0.75 to $3.79 for 1 participant ... refused before it starts if the expected cost plus a 10% margin is over $3"; study check --json's analysis.expectedCostUsd.high was 3.7892 (4.0392 for 6 participants). With a cap that refuses even a run with no evidence, it still printed a range. doctor --study's post-run analysis row and analyze --help's --max-cost stated the expected cost plus the margin alone. |
"expected $0.75 to $2.83 for 1 participant ... refused before it starts if both its worst case and its expected cost plus a 10% margin are over $3"; high 2.83079 (2.99999 for 6). With such a cap: "refused before it starts for 1 participant even with no evidence (expected $0.75), since both its worst case and its expected cost plus a 10% margin are over $0.5", and analysis.refusedFromUsd 0.7532 in place of analysis.expectedCostUsd. Doctor and the help state the same rule. Admission itself is unchanged. (#1735) |
Read analysis.refusedFromUsd when analysis.expectedCostUsd is absent. |
| 8 | An in-process run of a local-app study recorded subject.source: "app-url" in run.json, the JSON result and lanes[].subject. |
It records "local-app". npm 0.117.0 cannot read such a run: its verify grades it blocked ("one or more run.json sections have the wrong shape") and its review exits 2 with HUMANISH_INVALID_RUN_BUNDLE. The exported RunBundle type's subject.source gains "local-app". (#1733) |
Read both values. Open a local-app run with 0.118.0 or later. |
| 9 | study show --json and parseStudy filled a shared-world study's omitted execution.concurrency with its participant count ("concurrency": 2 for a host and a guest). |
The parsed config carries execution.concurrency only when the study declares it; the planner resolves it from the participants and the limit in row 2 (a dry run of 21 members prints participants: 21, up to 20 at once). (#1739) |
Read how many run at once from the plan, or declare it. |
| 10 | verify counted a folder it could not list, or the folder where the scan stopped at 10,000 entries, as a file: "cannot read 1 file as text ...: extras/many/ (the scan stops after 10000 entries)". |
"The public-safety scan stopped at its limit of 10000 files and folders while listing extras/many/, so it read nothing listed after that point.", and its own sentence for a folder it could not list. The grade is local_only on both. (#1733) |
Match UNSCANNED_ARTIFACT by its code. |
| 11 | A desktop-cli dry run's bundle summaries and participant sentences named an empty URL: "Dry run: 1 participant would use . Nothing was launched and $0 was spent." and "would use ; no session ran." | "Dry run: 1 participant would use humanish (desktop-cli). ..." and "would use humanish (desktop-cli); no session ran." (#1733) | Nothing. |
| 12 | In humanish tui, the legend kept "Enter set up humanish here" or "Enter select esc back" while an action other than a live start was armed; a study's median left out analyses (try-live, $0.23 run plus $0.62 analysis: "~$0.23 median"); Esc on the keys screen after entering a key went to the studies list. |
The legend reads "Enter confirm esc cancel" while armed, or "esc keep analyzing" for Cancel analysis; the same study reads "~$0.85 median"; Esc returns to the screen the keys were opened from. (#1735) | Nothing. |
Added
HUMANISH_E2B_MAX_CONCURRENT_SANDBOXEStells humanish how many sandboxes your E2B plan runs at
once: 20 when unset (E2B Hobby), 100 on Pro. A hosted computer-use or shared-world study runs at
most that many desktops at once, counting a provisioned shared world's app sandbox, and its plan
records a warning naming the setting when the limit holds participants to waves. An
execution.concurrencyabove the limit is refused before any sandbox is created, with the
setting value that would admit it.doctorshows the limit on ane2b concurrent sandboxes
row. A local desktop study keeps its own capacity checks (rows 2 and 3). (#1739, from #1737)
Changed
- A computer-use study may have up to 100 participants, up from 16, and a shared-world study is
held to the same 100; npm 0.117.0 refused a shared-world study of more than 16 members with the
computer-use refusal. The old refusal said every participant runs at once and no setting raises
the cap; how many run at once is now the E2B plan's limit above, andcaps.maxUsdbounds spend.
The new refusal says the 100 keeps one run's bundle and Observer view a size humanish supports. A
participantsgroup whosecountwould pass 100 is refused before it is expanded, and a
--countabove 100 before any participant is built (row 1). (#1739, from #1737) - A live run of more than 16 participants records its automatic analysis as skipped with
AUTOMATIC_ANALYSIS_PARTICIPANT_LIMIT, andrun,study check,doctorand the TUI say before
the run that it will not run. One analysis reads at most 16 participants; it would have analyzed
the first 16 and reported a partial result. The skip does not fail a run whose study declares
noreview.analysis; with a declaredreview.analysisthe run exits 2, as for any analysis it
asked for and did not get (row 4). (#1739, from #1737) - A shared-world study's omitted
execution.concurrencyis no longer filled with the participant
count by the parser. The planner resolves it, from the participants and the E2B plan's limit,
and a declared value above the participant count is clamped to it, as on computer use (row 9).
(#1739) - A study with real email receiving and more than 64 participants is refused when it is read. Its
run used to fail at start withcomms_authority_unavailable, because one run leases at most 64
inboxes (row 1). (#1739) - An in-process run of a
local-appstudy recordssubject.source: "local-app"in run.json, the
JSON result and each participant'slanes[].subject. It recorded"app-url".verify,
observe,reviewandexportread both values, so runs saved withapp-urlstill open; npm
0.117.0 cannot read a run that recordslocal-app. An in-process run of anapp-urlstudy
still recordsapp-url(row 8). (#1733, fixes #1716)
Fixes
humanish study checknow plans the study the wayhumanish run <study>does and fails a study
the run refuses before it starts, such as a computer-use study whose sandbox deadline passes
HUMANISH_E2B_MAX_SANDBOX_MINUTES, or alocal-appstudy, which only the library can run. The
failedplanrow and the error carry the run's code and message, and no reachability probe
runs. It passed such a study before. A study that plans gets the same output as before (row 5).
(#1733, fixes #1707)- A shared-world study, or a scripted study with a
clonesubject, whose sandbox deadline passes
the sandbox ceiling is refused before it starts, as computer-use and terminal studies are, with
theHUMANISH_E2B_MAX_SANDBOX_MINUTESvalue that would admit it. The code is
HUMANISH_SHARED_WORLD_INVALIDon shared world and the newHUMANISH_SCRIPTED_SUBJECT_INVALID
on the scripted route, which also uses it to refuse a value of the setting it cannot read. The
deadline is the subject sandbox's on a clone or local-tree subject (the session, 30 minutes to
provision, each seed step's budget and 10 minutes of teardown) and each participant's sandbox on
a shared-worldapp-urlstudy (the session and 10 minutes). Withexecution.timeoutMsset, or
with seed steps that leave no session under the ceiling, E2B refused the sandbox after the run
started. A scriptedapp-urlstudy creates no E2B sandbox and is not checked (row 6). (#1733,
fixes #1707) humanish verifynames a folder the public-safety scan could not list, and the folder where it
stopped at 10,000 files and folders, in their own sentences of theUNSCANNED_ARTIFACTreason:
"The public-safety scan stopped at its limit of 10000 files and folders while listing
extras/many/, so it read nothing listed after that point." It counted either one as a file it
could not read: "cannot read 1 file as text ...: extras/many/ (the scan stops after 10000
entries)" (row 10). (#1733, fixes #1691)- A
desktop-clidry run names the product the way thesubject:line does: "Dry run: 1
participant would use widgetsmith-cli (desktop-cli).", "would each use their own copy of
widgetsmith-cli (desktop-cli)" on a fan-out, and "would use widgetsmith-cli (desktop-cli); no
session ran." for each participant. The three sentences named an empty URL (row 11). (#1733,
fixes #1716) - A
participantslist entry with a very largecount, such as1000000000, no longer exhausts
memory while the study is read. The parser expanded every group before checking the roster's
size (row 1). (#1739) - The analysis line that
humanish run,study checkand the TUI print before a live run gives
the expected cost of the analysis admission would start. Its range ended at a run that keeps
evidence up to the limits, which the default $3 cap refuses: a one-participant study read
expected $0.75 to $3.79and now readsexpected $0.75 to $2.83. The line states the rule as
admission applies it: the analysis is refused when both its worst case and its expected cost
plus a 10% margin are over the cap.humanish doctorand the help foranalyze --max-costnow
state it in the same words; both said the expected cost plus the margin alone. When the cap
refuses even a run that keeps no evidence, the line says so, andstudy check --jsongives that
run's expected cost asanalysis.refusedFromUsdin place ofanalysis.expectedCostUsd(row 7).
(#1735, from #1720) - In
humanish tui, the key legend says what Enter and Esc do while an action waits for its
second Enter:⏎ confirm esc cancel, oresc keep analyzingfor Cancel analysis. While Stop,
Cancel analysis, Run again or Set up humanish here was armed, it still read⏎ select esc back
or⏎ set up humanish here(row 12). (#1735) - In
humanish tui, a study's median cost includes each run's analyses, as its run rows and run
cards do. A study whose two runs cost $0.55 and $1.06 with their analyses showed~$0.02 median, the participants and desktops alone; it now shows~$0.80 median(row 12). (#1735,
from #1720) - In
humanish tui, Esc on the keys screen after entering a key goes back to the screen the keys
were opened from. It went to the studies list, because the surface mounts again after the
hidden prompt and started over (row 12). (#1735, from #1720)
Known gaps
- Studies past 16 participants are the first slice of #1737. Still in progress there: participants
arriving over time (every participant starts at once, up to the concurrency limit), one analysis
of a run past 16 participants, and the Observer at 100 participants. (#1737) - A run whose automatic analysis was skipped for its participant count has no analysis record in
its bundle.reviewsays "No analysis has run for this run." and suggestsanalyze --run <id> --max-cost 3, which keeps the first 16 participants' evidence and lists the rest as omitted.
findings, the TUI run screen and the Observer show no reason either. Only the run's JSON result
and its last CLI line name the reason. (#1739) rundoes not print the warning that namesHUMANISH_E2B_MAX_CONCURRENT_SANDBOXESwhen the
limit holds participants to waves; the run records it as astudy.warningevent, and the
terminal shows only the plan line "concurrency 20, 2 waves". (#1739)- A study of many participants prints per-participant lines: 24 participants gave
run79 warning
lines (window clipping, window fill, unblurred screenshots, slow browser start), andrunand
study checkeach gave 24 persona-background warnings for a persona with no background. (#1739) runprints its analysis line from the declared participant count, so a--countoverride does
not change it, and a--countabove 64 with real email receiving still fails at run start.
(#1739)- A held Enter in
humanish tuican still confirm an armed action on its first key repeat. The
400 ms floor ignores repeats that come sooner, and a system's initial repeat delay is often
longer (500 ms by default in GNOME). (#1730) - A
serve.startthat bash cannot parse still waits outreadyTimeoutMs. The scripted and
shared-world routes give a failed build their generic failure code; the message carries bash's
error. (#1734) - On a desktop with speech, heard speech reaches the participant only with a screenshot and the
guest holds 8 utterances between two screenshots, so a long wait in a talking call can overflow
it; a wait there stays 30 seconds unless the study raisesactor.maxWaitMs. (#1724) - Two contained-file races from #1669 remain. They need a second local process, running as the
same user, to swap a directory in the run between two filesystem calls: temporary-file cleanup
still unlinks by path (item 1), and the reviewer-notes listing still listsnotes/by path and
can follow a swapped directory (item 3). Storage validation and the scan still list each folder
whole before the 10,000-entry cap applies, verify has no total read budget, and a legitimate
sandbox-receipts.ndjsonover 32 MiB is refused. (#1669) - Coding agents still do not tell their person about
humanish tui. (#1701) - Path redaction in terminal transcripts still removes the text after a path up to the next space,
such as a line break and the next header aftercwd:. Nothing leaks; text a reviewer needs is
lost. (#1702) - A hosted computer-use participant cannot type keysyms outside US printable ASCII (
é,€) as
keys; it has to type them as text. (#1713) - The routes still take
E2B_DOMAINfromprocess.envon create and ignore one in
RunStudyOptions.env. (#1717) - From #1646, after #1684: a 4-byte value inside a longer base64 run is not found, two further
decodings at a marker edge are not read,RunSecrets.spans(the terminal route's per-chunk
replacement) has no final check, and a value that is part of a marker nests markers. (#1646) - While a live run works, its study screen in
humanish tuireads "1 run, none live". - Design findings cannot say that text or a click target is too small by measure. (#1638)
- The impressions ask describes its kinds in visual terms, which can steer a participant toward
visual critique whatever its persona. (#1654) - Nothing Mac-specific has run on a Mac: the Lima VM sizing and
runtime setup --memory --cpus.
Package
humanish 0.118.0 packs to 2.5 MB, 7.8 MB unpacked, 968 files (npm pack on release/0.118.0 at
eec3b71, shasum 609c6faa2561d81e434e57761478d4bcb9f18310). Against 0.117.0 it adds
dist/run/subject-name.js and .d.ts and dist/study/automatic-analysis-plan.js and .d.ts, and
drops nothing. The package exports only ., with the same 13 values and 31 types as 0.117.0 (pnpm api:proof); the exported RunBundle's subject.source gains local-app (row 8). Its dependencies
are the same as 0.117.0's. It needs Node 22.19 or newer, as before, and @e2b/desktop 2.3.2 or
newer as an optional peer.
Checked before release
A live smoke on a tarball packed from main d183d1a in fresh projects under /tmp with a fresh HOME
and npm cache, DO_NOT_TRACK=1, and @e2b/desktop 2.4.0 beside humanish. Keys came from the user
key store through key discovery, with XDG_CONFIG_HOME pointing at it; no --dotenv was passed.
The store's OpenAI organization is non-ZDR, so the OpenAI participant ran threaded; its E2B key is
on a Pro plan. Controls ran the same study files or commands on npm 0.117.0, or read copies of the
main runs. Priced spend: $1.88 for the smoke's live runs, all on main ($0.62 of it in try-live's
analysis), then $0.25 for the dogfood run and $5.38 for the benchmark.
init --yeswrites the same files as npm 0.117.0, andkeysprints the same.doctorprints the
same apart from the newe2b concurrent sandboxesrow: 20 at once unset, 100 from the setting,
and a failed row with exit 2 forlots, where npm 0.117.0 exits 0 with no row (row 3).doctor --study try-livealso states the analysis rule in the new words (row 7).- try-live, as
initwrites it: reached the goal in 9 participant turns, input growing every turn
from 2,888 to 16,441 tokens, then 16,801 for the impressions request; cached input on each turn
from the second was the previous turn's input minus 3. $0.23 run.runprinted "expected $0.75
to $2.83 for 1 participant ... refused before it starts if both its worst case and its expected
cost plus a 10% margin are over $3", where npm 0.117.0'sstudy checkprints "$0.75 to $3.79 ...
if the expected cost plus a 10% margin is over $3" (row 7). Its automatic analysis was admitted at
an expected $1.01 (admittedCostUsd$1.11, worst case $2.00, cap $3) and billed $0.62, with 2
findings and 3 design findings. It gave 6 impressions (unclear, unclear, unfinished, liked, liked,
liked). - A 24-participant computer-use study on a one-paragraph static page, 3-minute sessions,
caps.maxUsd0.3, noreviewblock (rows 1, 2 and 4). With
HUMANISH_E2B_MAX_CONCURRENT_SANDBOXESunset, the plan read "concurrency 20, 2 waves", E2B listed
at most 20 of the run's sandboxes (10 samples 3 s apart at 20), participants 21 to 24 started as
earlier desktops ended, and the run recorded thestudy.warningnaming the setting;rundid not
print it. With the setting at 100, the plan read "concurrency 24, 1 wave" and E2B listed 24. Both
runs: 24 of 24 reached the goal, "analysis: not run, because the study has more than 16
participants and automatic analysis reads at most 16", exit 0, $0.50 and $0.54, 112 s and 70 s. study checkandrun --dry-runof three studies past the 60-minute ceiling (rows 5 and 6): a
computer-use local-tree study with a 90-minute deadline, which npm 0.117.0'sstudy checkpasses
and itsrunrefuses; a shared-world study with a 65-minute participant deadline and a scripted
clone study with a 70-minute subject deadline, which npm 0.117.0 plans and dry-runs. Main fails
all six withHUMANISH_COMPUTER_USE_SUBJECT_INVALID,HUMANISH_SHARED_WORLD_INVALIDand
HUMANISH_SCRIPTED_SUBJECT_INVALID. A shared world of 20 members (a provisioned clone) plans as
"up to 19 at once", one of 21 (a host and 20 guests on an external page) as "up to 20 at once",
and npm 0.117.0 refuses both at 16. Acount: 1000000000group is refused in 0.8 s; npm 0.117.0
ran out of memory after 44 s (row 1).- A library dry run of a
local-appstudy recordslocal-app; npm 0.117.0'sverifygrades that
runblockedand itsreviewexits 2 (row 8). verify,reviewandruns(text and--json) on copies of the three live runs: byte-identical
on main and npm 0.117.0, includingreviewof the 24-participant runs.- The TUI under a pseudo-terminal sized on that pty only, replayed through
@xterm/headless: with
Set up humanish here armed, the legend read "Enter confirm esc cancel" on main and "Enter set up
humanish here ..." on npm 0.117.0; Run again armed on try-live's finished run read "Enter confirm
esc cancel" and Esc cleared it with no run started. After a synthetic E2B key was entered at the
hidden prompt into a scratch config folder, Esc on the keys screen returned to the study it was
opened from on main and to the studies list on npm 0.117.0. On copies of the live runs, try-live's
median read "$0.85" on main and "$0.23" on npm 0.117.0 (row 12). - After every run,
reclaim --checkexited 0 withclean. Raw sandbox ids appear only in
sandbox-receipts.ndjson, E2B listed 0 tagged sandboxes before the smoke, after it and after the
benchmark, and no key value from the store appears in any run or kit file, as written, base64 or
hex (11,570 files).
On the release branch:
pnpm release:checkon the release head 30522ad, which adds the benchmark result:
exit 0 in 3.8 min with the same counts as below, a public-surface scan of 3,048 text files and the
same shasum. On the release commit eec3b71 before it: exit 0 in 6.2 min, 507 test files passed
and 10 skipped, 7,981 tests passed and 12 skipped; TUI 192 tests; lint 426 warnings at its cap of
426; api:proof "13 values and 31 types match"; public-surface scan of 3,046 text files and 527
binary assets;npm pack --dry-run968 files with shasum
609c6faa2561d81e434e57761478d4bcb9f18310.pnpm docs:generatethere wrote 0.118.0 into cli.mdx.pnpm release:dogfoodon eec3b71 (the script run with Node, keys read from the user store by
Node's--env-fileinto its environment, and a temporary HOME): "release:dogfood ok", verdict
blocked(blocked_approval), no-spend satisfied, $0.25. The participant raninit, the free
four-participant dry run,verify("Passed all 16 verification checks; evidence was
share_ready") andreviewon the installed 0.118.0, and stopped at the keys and the local VM
devices the gate withholds; the script reads that verdict as the expected stop.verifyon a copy
of the run:share_ready, 16 of 16;reclaim --checkclean.- The post-publish check on the release tarball from eec3b71 (
TMPDIR=/tmp,HUMANISH_SPECthe
tarball): 117 of 117 passed. Its 6 new rows cover doctor's concurrent-sandboxes row at the
default, 100 andlots(#1739); a 17-participant dry run with the analysis skip instudy check
and 101 refused (#1739); anexecution.concurrencyof 24 refused at 20 naming the setting and
planned at 100 (#1739);study checkfailing a 90-minute deadline (#1733);subject.source
local-appfrom a library dry run (#1733); and verify's 10,000-entry sentence (#1733). Rows 20bd
and 20bj now expect the new analysis line, and 20br the new verify wording. The same script on npm
0.117.0 passed 109 and failed the 6 new rows and those 2 analysis-line rows. A rehearsal on the
smoke's main tarball passed 114 of 117; its failures led to the 20bd and 20bj changes and fixed
my own check in the new row 20ca. - The consumer check: the 0.117.0 check's library and CLI consumer, plus 14 CLI lines for this
release. On the release tarball the consumer typechecks with 0 errors, as on npm 0.117.0, and
needs no migration. The library steps are the same on both versions, and of the 110 CLI lines from
the 0.117.0 check, 107 are the same; the three that differ are the six-participant analysis range
in text and JSON ($4.04 to $3.00, row 7) and verify's 10,000-entry sentence (row 10). The new lines
differ as rows 1 to 8 say. pnpm benchon eec3b71 with its defaults (3 runs per arm,openai-computer-use,neutral
mission), committed in 30522ad: report recall 6/15, analysis recall 6/15, 0 invented on either
arm, $5.38 estimated. 0.117.0 scored 8/15 and 7/15, and 8/15 and 8/15 read by hand. The two
unresolved analysis lines are on the clean arm and describe what the clean build does, so recall
read by hand stays 6/15 and 6/15. D1 and D5 account for the drop: only one planted participant
pressed Clear completed (3 of 3 on 0.117.0), on the empty list'sundefinedrow before adding a
task in the same turn, and its report says "Clear completed removed it", which the rubric counts as
the one false assurance; all three planted participants used Save in edit mode and none reported
it failing, where one report and one analysis did on 0.117.0. D3 and D4 stayed 3/3 for reports and
analyses. The analysis cap was the dry run'sadmittedCostUsd, $1.13, against bills of $0.53 to
$0.75; none was refused or skipped. No planted participant typed more than 30 characters in one
action, so none met D2. Every participant ranthreaded, ended on its own and gave 5 or 6
impressions.
Not verified
- macOS and Windows. Every check ran on Linux x64 with Node 24.12.
- A live run on the published package: the live smoke ran on a main tarball, and the post-publish
rows are offline dry runs, refusals and local runs. - A live run on an E2B Hobby key, and E2B's own refusal past a plan's concurrency limit: the store's
key is on Pro, so the 20-at-once limit was humanish's own. - A live run of more than 24 participants, a live shared-world run of any size, and a
--count
override past 16 on a live run. - A live run over 16 participants whose study declares
review.analysis, which exits 2; tests
cover it. The run result'sAUTOMATIC_ANALYSIS_PARTICIPANT_LIMITwas not read from a live run's
--jsonoutput; the smoke saw the CLI line and the bundle holds no analysis record. analyze --runon a run over 16 participants: a dry run admits it at an expected $2.21, and
src/analysis/evidence.tskeeps the first 16 participants' evidence; no request was sent.- A shared-world or scripted clone study refused at the ceiling on a live run: all checks were at
plan time, and no sandbox was created. - Real email receiving past 64 participants: tests only.
- In the TUI: the "esc keep analyzing" legend on Cancel analysis (the analysis finished before the
arm), the Stop legend on a working run, the AgentMail path after key entry, and the median on a
study with more than one analyzed run. The TUI tests cover them. - An in-process
local-apprun that is not a dry run.