Repository navigation
0.133.0: An OpenAI participant names how long it waits
humanish 0.133.0 lets an OpenAI computer-use participant name how long it waits, with a
wait_minutes tool, and cuts what a participant costs while nothing on its screen changes. A turn of
only waits is watched, and each further quiet wait in a row lasts twice as long, starting from the
length the participant named, up to 10 minutes. In live 60-minute runs against 0.132.0, median model
spend per participant-hour fell 13% on a resident portal, 52% on a support queue, 46% on an issue
list that shows new issues only on reload, and 78% on a dispatch board, and all 6 participants on
the issue list reloaded and reported every new issue. verify and analysis also no longer fail on a
run whose traces together pass 512 MiB.
Added
- An OpenAI computer-use participant can name how long it waits, as Codex and Claude Code participants do. Every request, from the first, offers a
wait_minutesfunction tool with one field,minutes, next to the computer tool, so the cached prompt prefix does not change. Its description is "Wait, then get a new screenshot. The reply says how long the wait lasted and why it ended." A call in a turn of only waits is watched for its minutes: it ends early when a change on the screen holds still, and lasts at mostactor.maxWaitMs, or 10 minutes when the study does not set it. The named minutes set where the doubling of quiet turns in a row starts: a participant that names 1 minute on each turn of a quiet screen waits 60, 120, 240 and 480 seconds, then 10 minutes at a time, and any other action, such as a reload, starts again from 1 minute. Minutes that are not a finite number above zero make a wait with no length, as the computer tool'swaitis. A call to a function the participant does not have is answered that there is no such tool and is a wait with no length, so a reply of only such a call does not end the session. A call sent with other actions goes to the desktop for at mostactor.maxWaitMs, as any such wait does. Each wait'sui_actionin the trace carriesnamedMswhen the wait named its length, before any cap. In a live A/B of 60-minute runs withgpt-5.6-solagainst 0.132.0, 3 runs per arm on each page and 6 on the reload page, median model spend per participant-hour fell from $0.92 to $0.81 on a resident portal that shows a notice and a badge, from $1.36 to $0.66 on a support queue, from $4.77 to $2.57 on an issue list that gains issues only on reload, and from $6.86 to $1.50 on a dispatch board. Every scheduled change reached a request, and all 6 participants on the issue list first reloaded within 2.4 minutes and reported all four new issues (#1862).
Changed
- A turn of only waits that asks for some wait time is now planned to last at least
idleWaitMs, for OpenAI, Codex and Claude Code participants, so naming 1 ms on each turn of a quiet screen waits 5, 10 and 20 seconds by default. Before, a turn whose named waits came to 1 ms lasted 1 ms, then 2 ms, and sent its requests nearly back to back. The Codex tool description and the Claude Code prompt state the floor (#1862). - The note after a wait no longer says how long the next wait can last. In live hours on the issue list, OpenAI participants whose tool description and notes said each wait in a row lasts twice as long named 1, 2, 4, 8 and 10 minutes in turn. One of three never reloaded and reported that no new issue appeared when four had; the other two first reloaded after 18 and 26 minutes. The note still says how long the wait lasted and why it ended. This also removes the note's misleading length after a small change ended a wait early (#1869).
- The Codex tool description and the Claude Code prompt no longer say that a turn of only waits returns when the screen changes. They say how such a turn's ask is counted, and the note after it says how long it lasted and why it ended. A tool named
wait_for_changeand described that way led OpenAI participants to wait for the issue list to change: in their first reasoning, 1 of about 10 planned to refresh the page, against 7 of 8 on 0.132.0, and 3 of 9 reported fewer issues than had arrived. The new Codex and Claude Code wording has no live run yet (#1862).
Fixes
- verify scans the decoded run one participant at a time. It scanned run.json with every participant's trace loaded as one string, and V8's longest string is 536,870,888 characters, so on a run whose traces together passed about 512 MiB, such as 25 participants at 30 MiB, verify stopped with
RangeError: Invalid string lengthand analysis withANALYSIS_UNAVAILABLE. On 25 synthetic participants with 30 MiB traces, verify now grades the runshare_readyandanalyze --dry-runpasses. A key planted in the last participant's trace, escaped so that only the decoded run shows it, still blocks the run. The scan answers as the single string did on every value JSON.parse returns. One pattern can match across participants: a URL whose user info runs up to an@and an E2B host. verify checks it across each cut and keeps only the URL's scheme between participants. Peak memory on that run was 2,044 MiB; verify still holds every trace at once. (#1848) - The bundle estimate warns when all participants' traces together may pass 512 MiB. The Observer page and the Observer server join every participant's items in one string, so on the 25 x 30 MiB run
humanish observestill stops withRangeError: Invalid string length. On a hosted desktop, the sandbox id scrub joins them too, on each write of run.json, so a hosted run that size stops updating run.json once its finished participants' traces pass the limit (#1870).humanish export --format bundlejoins them as well, once--max-bytesis raised past its 25 MiB default. The Observer fix is #1871. The estimate reaches the limit at 40 participants working 8 hours. It warns about the Observer page passing 32 MiB from 20. (#1848)
Measured before and after
A live A/B of 60-minute runs, gpt-5.6-sol medium, threaded context, against 0.132.0 run at the same time on four synthetic pages: 3 runs per arm on each page and 6 on the issue list.
| Page | 0.132.0 $/h | 0.133.0 $/h | Requests per hour |
|---|---|---|---|
| Resident portal (a notice and a badge appear by themselves) | 0.92 | 0.81 | 40 -> 36 |
| Support queue (rows appear by themselves) | 1.36 | 0.66 | 51 -> 32 |
| Issue list (a new issue shows only after a reload) | 4.77 | 2.57 | 114 -> 74 |
| Dispatch board | 6.86 | 1.50 | 130 -> 54 |
Every scheduled change reached a request in every run on the three pages that update by themselves. On the issue list, all 6 participants first reloaded within 2.4 minutes and reported all four new issues. Three earlier rounds settled the wording: a tool named wait_for_change, described as ending when the screen changes, led participants to wait for the list to change instead of reloading it, and 3 of 9 reported fewer issues than had arrived.
On the release tarball, a 15-minute study of one participant watching a static page until its scheduled end asked wait_minutes for 5, 10 and 4 minutes, ended on schedule, cost $0.08, and verified clean. The efficacy benchmark on the release commit scored report recall 9/15 and analysis recall 8/15 (9/15 read by hand), with nothing invented on either arm; see the benchmark README.