Repository navigation
0.154.0: Day-long studies with arrivals run remotely and show live status
humanish 0.154.0 fixes what an 8-hour study of 20 participants on a public movie game turned up.
A roster whose participants arrive over the day runs remotely when the most it runs at once fits
your E2B limit, and while it runs each participant reads as queued, running or ended with its
outcome, in run.json and the Observer. A participant on an E2B or Modal desktop keeps going after
one dropped connection on a screenshot or a type that did not finish. Ctrl-C after a remote run
has finished keeps its result. The study is recorded in
docs/evidence/computer-use/full-day-study-2026-10-28.md.
Fixes
- Ctrl-C on a remote run whose participants have finished stops only its analysis, says so, and
exits with the run's own result. Before, it printed that it was stopping the run and reclaiming
its participants, which had already ended, and exited 130 while the result saidok: true. - While a run is going, a participant whose start is still ahead is
queued, and the Observer
card says when it starts ("Starts at +2h 35m."). It turnsrunningonce its session starts.
Before, every participant read as running from the first minute, and the Observer said "20 of 20
participants active" while 2 had started. The subject line inevents.ndjsonno longer calls a
public app URL loopback. - While a run is going, a participant whose session ended shows its outcome: its status, summary
and session event inrun.jsonandevents.ndjson, and its Observer card. Before, it read as
running until every participant had ended, so one that left at minute 15 of an 8-hour run was
counted as active for 7 more hours. - A hosted study whose participants arrive over time runs remotely when the most participants its
schedule runs at once, plus the runner, fit your E2B concurrent sandboxes. Before, a roster of 20
arriving over 8 hours, at most 4 at once, counted as 20 at once and ran on this machine. The run
in an E2B runner now holds its desktops to your limit less the runner. - A computer-use participant on an E2B or Modal desktop whose
typedid not finish keeps going.
The action is reported to its model as a failed desktop command, with a note that the field may
hold part of the text, and is not retried. Before, the participant ended withactor_error. - A click at x 0 or y 0 on an E2B or Modal desktop moves the pointer there first. Before, it
clicked wherever the pointer was, while the trace recorded the edge coordinate. - A computer-use participant whose screenshot fails because the connection to its desktop dropped
(fetch failed, a reset socket, or gRPC UNAVAILABLE) is given a second look 3 seconds later, with
a warning in its trace. A second failure still ends it. Before, one dropped connection ended an
8-hour participant withactor_error3 hours 42 minutes in. - A plan whose participants include some with their own end or cap says so: "at most 4 run at once
when every session uses its full budget: its own end or cap, or else 40m", and the fan-out line
gives the study budget "per participant without its own end or cap". Before, a roster with two
8-hour participants read as if every session used the 40-minute budget, though the figures
already counted the 8 hours.
Benchmark
The Taskly benchmark's gpt-6.1-sol arm, 12 planted and 12 clean runs on the in-memory build, ran
on the release candidate: report recall 26/60 (30/60 on 10-27), analysis recall 34/60 (27/55), and
nothing invented on the clean arm. See docs/evidence/benchmark/README.md.