Repository navigation
0.134.0: Participants held to an end keep reloading until it
humanish 0.134.0 keeps a participant held to a scheduled end running when it reloads a page that
changes only on reload. 20 reloads in a row on an unchanged screen ended its session as gave_up.
Now humanish watches the screen after each further reload, as after a wait, so a participant that
reloads every few seconds costs about what one that waits costs. In 60-minute live runs with a
persona that refreshes every few seconds, 0.133.1 ended 3 of 4 within 7 minutes; this release ran
all 4 to their end at a median $3.23 an hour. Trace files are now written as compact JSON, about a
quarter smaller, and a run whose trace still passes the 32 MiB read limit finishes and grades
local_only where it failed.
Fixes
- A participant held to a scheduled end (
endAfterMs) is no longer ended asgave_upfor reloading a page that changes only on reload. 20 repeats in a row of a recent action on an unchanged screen ended its session: in one of 72 live 60-minute runs on 0.132.0, a participant that reloaded an issue list every 9 seconds was ended at minute 13. Such a streak now ends only a participant without a scheduled end, and a held one whose 20 repeats never reached the desktop, as when each reload's desktop command fails. From the 20th repeat on, humanish watches a held participant's screen after each further repeat before its next request, as after a wait with no length: 5, 10 and 20 seconds, doubling up toactor.maxWaitMs. The trace records the streak in arepeated an action on an unchanged screennotice with its turn count and time span, and review.json lists it in its gaps. In 60-minute live runs with a persona that refreshes every few seconds, 0.133.1 ended 3 of 4 asgave_upwithin 7 minutes; this release ran all 4 to their end at a median $3.23 an hour (#1873). - A participant held to a scheduled end no longer sends model requests back to back with turns of only screenshots or only 0 ms waits. Such a turn after another turn of only waits or screenshots is watched as a wait with no length,
idleWaitMsand doubling, and its 0 ms waits are not sent to the desktop. The first such turn, at the start or after an action, goes out at once. A watch that no wait carries is recorded on awatched the screennotice with the usualwatchfield. Participants without a scheduled end keep their behavior (#1868). - A run whose participant's trace passes 32 MiB, the most humanish reads of one file, now finishes. The sandbox id sweep at the end of the run threw when it could not read the trace. It now records each file it could not read in a
sandbox-ids.unresolvedwarn event in run.json, and the run goes on.humanish verifylists such a trace underUNSCANNED_ARTIFACT, grades the runlocal_onlyand names the trace in its warnings, where it reported the run invalid and exited 2. The Observer shows that participant without its items, andhumanish analyzerefuses the run asANALYSIS_SOURCE_TOO_LARGE, naming the participant and the file (#1877).
Changed
- Trace files are written as one line of compact JSON. Indented, a trace was 1.36 times its compact size over 236 kept computer-use traces, so a participant now reaches the 32 MiB read limit about 1.36 times later. A synthetic participant with 30 MiB of items, repeated from a recorded participant of 78 turns, wrote a 47,068,524-byte trace before and a 34,552,066-byte trace now. run.json stays indented. The bundle estimate counts 3.5 KB of trace a turn where it counted 4.7 KB, so a participant at the busiest measured rate passes the trace limit only after about 27 hours, longer than a window may last (#1877).
Live check of the held-participant change
22 runs of 60 minutes on gpt-5.6-sol, 0.133.1 against the change, on an issue list that gains four issues, shown only after a reload:
| 0.133.1 | 0.134.0 | |
|---|---|---|
Persona that refreshes every few seconds: ended gave_up on the streak |
3 of 4, at 4 to 7 minutes | 0 of 4 |
| Same persona: reached its scheduled end | 1 of 4, $4.89 an hour | 4 of 4, median $3.23 an hour |
| Reader persona: median spend; reported all four issues | $3.17 an hour; 4 of 4 | $2.72 an hour; 4 of 4 |
| A page the participant acts on: median spend; every change reached a request | $0.83 an hour; yes | $0.95 an hour; yes |
One refreshing participant on 0.134.0 reported only the first issue. It injected its own refresh script into the page at minute 11, and the script never showed the later issues; the 0.133.1 refresher that reached its end also wrote DevTools scripts.
The efficacy benchmark was not rerun. Its participants have no scheduled end, so the participant changes in this release do not reach them, and the trace change alters the file's formatting only. The 0.133.0 result stands.