Repository navigation
0.127.0: Clock-end passes counted by what participants declared, a scaled wind-down, the concurrency cap in the plan
humanish 0.127.0 makes long scheduled studies report what happened. A participant that passes at
its scheduled end or session cap without declaring REACHED THE GOAL. is now counted as running
until that end (participants.ranUntilEnd), where 0.126.0 counted it as reaching the goal. The
default wind-down is a quarter of the window, at most 10 minutes, so a short window is no longer
told its end from its first turn. HUMANISH_CUA_MAX_CONCURRENCY now lowers the plan itself, on
shared-world studies too, and a participant whose scheduled end passes while it waits for a slot
is skipped before its desktop is created. The Observer keeps updating past 100,000 items for a
participant, and the bundle estimate is re-derived for run bundle v3.
This is a minor release because four behaviors change:
| # | 0.126.0 | 0.127.0 | What to do |
|---|---|---|---|
| 1 | A participant that passed at its scheduled end or session cap counted in the review's participants.reachedGoal and was described as reaching the goal, whatever its closing report declared. |
Only a declared REACHED THE GOAL. counts in reachedGoal. Others count in participants.ranUntilEnd (present when above 0), and the summary says 1 ran until its session cap or 1 ran until its scheduled end. Older bundles keep their counts when re-read. (#1816) |
A script that reads reachedGoal for clock-end studies also reads ranUntilEnd. |
| 2 | With no windDownMs, a participant was told the time left for the last 10 minutes of its window. |
The last quarter of its window, at most 10 minutes. (#1816) | Set windDownMs: 600000 to keep 10 minutes on a short window. |
| 3 | HUMANISH_CUA_MAX_CONCURRENCY lowered only the run's participant table, and did not apply to shared-world. |
It lowers the plan, study check and the run, before the E2B plan's limit, on shared-world too; a value under 2 refuses a shared-world study. |
Unset it or set it to 2 or more for shared-world studies. |
| 4 | A participant with endAfterMs that waited for a slot past its end created a desktop, gave a closing report at its first turn and passed. |
It is skipped before its desktop is created and recorded blocked with the reason; the plan warns, naming it. |
Let more participants run at once, or give it a later end. |
Changed
- A computer-use participant with
endAfterMsormaxSessionMsand nowindDownMsis told the time left for the last quarter of its window, at most 10 minutes. The default was a fixed 10 minutes, so a window of 10 minutes or less was told its end from its first turn. A 10-minute cap is now told from 7 minutes 30 seconds in, and a window of 40 minutes or more keeps 10 minutes. An explicitwindDownMsis used as written.study check'swindowsrow and the plan entry'swindDownMsshow the new default. (#1816) - The
participants[].filesrefusal messages suggestfixtures/photo.pngandDownloads/photo.png, and the study-file docs examples use neutral roles (support agents, customers, applicants). (#1822) HUMANISH_CUA_MAX_CONCURRENCYnow lowers the plan itself.study check'sschedulerow, the plan's late-start warnings and the run'sschedule:line use the lowered count; before, only the run's participant table did, sostudy checkcould describe slots the run did not have. The plan applies it before the E2B plan's limit, so anexecution.concurrencyabove that limit is no longer refused when the setting lowers it. A lowering adds a plan warning, as the E2B plan's limit does; when that limit lowers the count further, the plan carries its warning instead. The setting now applies toshared-worldstudies too, and a value under 2 refuses one, since a shared world runs at least two participants at once. (#1826)
Fixes
- A served Observer page keeps updating once a participant passes 100,000 items. Past that count it refused every feed body and kept the last one it accepted, through the end of the run, so a later generation of a live file that scrubbed a sandbox id the run learned late never reached the page. A page opened after that count showed no run at all. The page no longer refuses a body for the length of a list. The server still reads a participant's trace or live file only up to 32 MiB; past that the page shows the participant's trace as unavailable, with a warning. (#1806)
- A computer-use participant with
endAfterMsthat waits for a free slot until its end has passed is skipped before its desktop is created, and its record isblockedwith the reason. Before, it created a desktop when a slot freed, was asked for its closing report at its first turn and passed. The plan gives such a participant no slot when it works out the schedule, so the participants after it start sooner in the plan, and it warns, naming each one it would skip. (#1826) - The bundle estimate that
study check,humanish runanddoctor --studyprint is re-derived for run bundle v3 from live and dry runs measured on 0.125.0. It checks run.json at about 225 bytes a model turn against the 16 MiB analysis reads, and each participant's trace against the same limit at 4.7 KB a turn, the busiest participant measured; the average was about 2 KB. It checks the Observer page against the 32 MiB verify reads, as the app, half of run.json and 0.12 of all the traces' bytes, and the file count against the public-safety scan. It no longer checks observer-data.json, which stays within 7 % of run.json. Five participants working 8 hours beside 25 short ones were estimated at a 30.2 MiB run.json and warned about; they are now estimated at 3.7 MiB. The warning starts at 17 participants working 8 hours for the Observer page, 30 for run.json, and a 10-hour window for one participant's trace. Thebundle sizerow names the largest trace and the Observer page in place of the largest file. (#1819) - A computer-use participant that passes at a scheduled end or a session cap without declaring
REACHED THE GOAL.is no longer counted or described as reaching the goal. Its closing report saidDID NOT REACH THE GOAL.or declared nothing. The review counts it in a newparticipants.ranUntilEnd, present when above 0, and its summary says1 ran until its session capor1 ran until its scheduled end. The participant's own summary says the same, and the Observer labels itRan until session caporRan until scheduled end. It still passes. A participant that declaredREACHED THE GOAL.still counts inreachedGoal. An older bundle keeps itsreachedGoalcount when re-read, and its completion line names the clock end. Analysis input still recordsgoalSource: participant_reportfor such a participant, as 0.125.0 did, so an analysis saved then stays valid; itsstopCauseanddeclaredOutcomename the clock end. (#1816) - The declared outcome skips a
>blockquote line. A closing report or last message whose first line quotes a page'sREACHED THE GOAL.no longer declares it; the blocker scan already dropped blockquote lines. (#1816)
Checked before release
- CI on this PR (sharded gate on both Node versions, site, Observer proofs, TUI, guest desktop, secret scan) is the gate of record.
pnpm gateon the release commit passed. - The post-publish check rehearsed on the release tarball (
pnpm packed:check): all 125 rows passed. The registry and unpkg rows run after publish. pnpm release:dogfood: a participant installed the tarball, ran the offline first-run study (share_ready, 16 of 16 checks, $0) and stopped at the credentials the gate withholds, which counts as passing.- Live on the release tarball (hosted E2B, Chrome, keys from the user store), about $0.66 estimated across three runs:
- try-live with analysis: finished, analysis complete with 3 findings and 3 design findings, verify 16 of 16 (
local_only, raw screenshots). HUMANISH_CUA_MAX_CONCURRENCY=1with two participants on example.com.study checkprinted the plan's warnings thatlatewould be skipped and that the 2 participants run 1 at a time, and itswindowsrow sayssteadyis told the time left from 1m 30s into its 2m cap. In the run,steadyworked until its cap, declaredREACHED THE GOAL.in its closing report and passed withstopCause: session_cap.late(endAfterMs: 60000) was skipped before any desktop was created: run.json records itblockedwith the reason, and the run's only sandbox receipt issteady's. The run failed as 1 of 2 passed, as a skipped participant does. verify 16 of 16.- One participant held to
endAfterMs: 120000that read the page early and watched until its end: passed withstopCause: scheduled_end. Its closing report declared no outcome, so the review counted it inranUntilEndand said "1 ran until its scheduled end", withreachedGoal: 0. verify 16 of 16.
- try-live with analysis: finished, analysis complete with 3 findings and 3 design findings, verify 16 of 16 (
- Each change had a Claude review; the Observer change had a Codex review with no findings. Every real finding is fixed with a test or listed below.
- The benchmark was not run: no change alters what a participant is asked or what the analysis reads.
Not verified
- The closing report at a clock end does not ask an OpenAI participant for its declared line, so a participant that reached its goal often lands in
ranUntilEnd(#1827). - A skipped participant's run line says "unknown ending" and the summary counts it as taking part (#1828).
- A live run past 100,000 items for one participant; analysis still refuses a run past 100,000 events or items (#1824).
HUMANISH_CUA_MAX_CONCURRENCYon a live shared-world run.- macOS and Windows.
Package
humanish@0.127.0: 1004 files, shasum cc9212c965f6e4a4711fc57396288dbb6ea77807.