Skip to content

0.122.0: Participants end on the clock, a harness error ends only its participant

Choose a tag to compare

@danielgwilson danielgwilson released this 12 Oct 01:20
· 184 commits to main since this release
d1e43da

humanish 0.122.0 lets a participant end at a scheduled time and keeps one participant's harness
error from stopping the rest of a study. participants[].endAfterMs, on the same clock as
startAfterMs, ends a participant's session at that time with stop cause scheduled_end,
counted as reaching the goal; from endAfterMs - windDownMs (10 minutes by default) each turn
tells the participant the end time and the minutes left, its turns that only wait no longer count
toward the idle limits before then, and its sandbox is sized from its own window. actor.clock { timeZone } adds the local time to each turn, and participants[].idleWaitMs sets one
participant's wait length, so staff on a long shift can check the screen less often than patients.
A participant's harness error now ends only that participant: the study-wide fail-fast is removed,
and every later participant starts on its schedule.

This is a minor release because it adds study fields and outputs change for the same input:

# 0.121.0 0.122.0 What to do
1 A computer-use participant's harness error tripped a study-wide fail-fast: every participant still waiting to start was recorded blocked with "a prior participant ended in a harness error (fail-fast)", and the run wrote a cua-lab.fanout.fail-fast event. Only that participant ends; later participants start on schedule. The fail-fast event is gone. A failure that never recovers (a rejected key) now ends each participant at its first model turn; caps.maxTotalUsd still skips participants due after the spend crosses it. (#1767) Expect more failures and fewer skips; stop reading the fail-fast event.
2 participants[].endAfterMs, windDownMs, idleWaitMs and actor.clock were refused as unknown fields. They are study fields; a participant with an end stops at it with stop cause scheduled_end, counted as reaching the goal. 0.121.0 refuses a study that sets them. (#1768) Read scheduled_end as a planned end.
3 A participant's sandbox deadline came from execution.timeoutMs for everyone. A participant with an end gets a deadline from its own window; study check prints each window. (#1768) Nothing.

Added

  • A computer-use participant can end at a set time: participants[].endAfterMs, on the same
    clock as startAfterMs, with windDownMs (default 10 minutes). From the start of the
    wind-down each turn tells the participant the end time and the minutes left. At the end the
    participant is asked for its closing report, and its session ends as passed with stop cause
    scheduled_end. Before its end, a turn that only waits counts toward neither the idle nor the
    no-progress backstop, so a staff participant can wait out a quiet shift. A participant that
    ends its own session before its end is told the time left and keeps going; repeated attempts
    to end count toward the idle backstop. Its session lasts its window plus 5 minutes in place of
    execution.timeoutMs, and its sandbox deadline is sized from that, so a short patient and a
    long staff shift can share a study. The plan refuses an end at or before the start, a window
    whose sandbox would outlive the sandbox ceiling, and an end on shared-world. study check
    prints each participant's window, and the plan line shows end=+<time> (#1764).
  • actor.clock: { timeZone } adds the local time in that time zone (a tz database name such as Europe/London) to each computer-use turn, and
    gives a scheduled end in that zone (#1764).
  • participants[].idleWaitMs, also on count: groups, sets one participant's idle wait over
    actor.idleWaitMs, so a staff member can wait a minute at a time while patients keep 5
    seconds. It takes the same range and is refused above an explicit actor.maxWaitMs; the
    participant's trace records it in waitSettings (#1764).

Fixes

  • One participant's harness error in a hosted computer-use study no longer skips the participants
    after it. The first harness error in a session tripped a study-wide fail-fast, so every
    participant still due by startAfterMs, or waiting for a free slot, was recorded blocked with
    "a prior participant () ended in a harness error (fail-fast)", and a 4-hour study lost
    everything after one desktop stall. A harness error now ends only that participant: the run
    records it as failed with its reason in its lane, its review gap line and its events.ndjson
    event, and every later participant starts at its time. The cua-lab.fanout.fail-fast event is
    no longer written. A clone, local-tree or desktop-cli subject that fails to set up in the
    first participant's sandbox still blocks the rest (#1765).

Checked before release

  • pnpm release:check on the release commit: exit 0; 8,099 tests passed, 12 skipped; TUI 192; lint 426 at its cap; pack 976 files, shasum f93bbc883535420da2c9932862216f955a240ea1. A first run failed one test file's cleanup (tests/substrates/detached-on-bash.test.ts, ENOTEMPTY removing a step folder a detached process was still writing); it passed 6 of 6 alone and the rerun of the whole gate passed. That test's cleanup is fixed after this release.
  • #1768: planner, loop and fake-desktop tests (a participant with an end waits 60 idle turns and ends at its end with scheduled_end; two participants record different idleWaitMs); three live hosted runs during the PR, the last ending its staff participant at its end with a closing report.
  • #1767: a fake-desktop run whose first participant stalls 10 minutes in starts every later arrival on time.
  • Live on the release tarball (hosted E2B, keys from the user store): two staff participants with a 3-minute endAfterMs and nothing to do, one at the default idleWaitMs (5 s) and one at 30 s, while two patients arrived at +30 s and +60 s. Both staff ended at their end with scheduled_end, passed (5 and 3 model turns; each trace records its own idleWaitMs); both patients reached the goal; exit 0; estimated $0.34. The model batched its waits (up to 32 wait actions in 5 turns), so this run did not approach the idle limits; the loop test covers that case.

Not verified

  • A live run of the Codex or local-agent participant with an end or a clock.
  • A 4-hour shift live; the longest live window was 3 minutes.
  • macOS and Windows.

Package

humanish@0.122.0: 976 files.