Skip to content

0.127.1: Clock-end closing reports declare their outcome; skipped participants named as skipped

Choose a tag to compare

@danielgwilson danielgwilson released this 13 Oct 07:52
· 161 commits to main since this release
9251af5

humanish 0.127.1 fixes two things the 0.127.0 release smoke found in scheduled studies. An OpenAI
participant asked for its closing report at a scheduled end or session cap now says how the task
went in a structured outcome field, so one that reached its goal counts in reachedGoal where
0.127.0 counted it in ranUntilEnd. A participant the run skips before its session starts now
shows its skip reason on its run line, and the summary counts it apart from those that took
part.

Fixes

  • An OpenAI computer-use participant asked for its closing report at a scheduled end or session cap now says how the task went in the report's outcome field (reached, not_reached or blocked). The closing request asks for it, and its strict JSON schema requires it. The trace's declaredOutcome reads the field before the summary's first line. Before, the request did not ask for the declared line, so a participant that reached its goal before its end counted in participants.ranUntilEnd. A provider port or stored trace without the field stays valid, and a report after a stop condition or dwell window still sets no declaredOutcome. A provider port's self-ending turn whose closingReport carries outcome sets declaredOutcome from it too. (#1827)
  • A computer-use participant the run skips before its session starts, because the study budget was reached, its scheduled end passed while it waited for a slot, or an earlier participant failed to provision its world, now ends its humanish run line on the skip reason. The line read blocked · unknown ending. The review summary in run.json and review.md counts skipped participants in a sentence of their own: a run of 3 with one skipped reads "2 participants took part, each in their own copy of the app: ... 1 participant was skipped before its session started; the gaps say why." It used to count all 3 as taking part. While a fan-out run is live, its summary says the participants "are planned, each in their own copy of the app". It said each was using its copy, which was untrue for one still waiting or already skipped. (#1828)

Checked before release

  • CI on this PR (sharded gate on both Node versions, site, Observer proofs, TUI, guest desktop, secret scan) is the gate of record. pnpm gate on the release commit passed.
  • The post-publish check rehearsed on the release tarball (pnpm packed:check): all 125 rows passed. The registry and unpkg rows run after publish.
  • pnpm release:dogfood: a participant installed the tarball, ran the offline first-run study (share_ready, 16 of 16 checks, $0) and passed.
  • Live on the release tarball (hosted E2B, Chrome, keys from the user store), about $0.46 estimated across two runs, the 0.127.0 smoke's B and C studies:
    • One participant held to endAfterMs: 120000 on example.com that read the page early and watched until its end: its closing report set declaredOutcome: reached from the new field, its summary has no declared first line, and the review counted it in reachedGoal. On 0.127.0 the same study counted it in ranUntilEnd. verify 16 of 16.
    • HUMANISH_CUA_MAX_CONCURRENCY=1 with two participants: steady worked until its 2m cap and declared reached; late was skipped before any desktop was created. Its run line reads participant late: blocked · skipped: its scheduled end (participants[].endAfterMs) came while it waited to start..., and the summary reads "1 participant took part, in its own copy of the app: 1/1 reported reaching the goal, 1 reported friction. 1 participant was skipped before its session started; the gaps say why." verify 16 of 16.
  • Before merge, each fix had a Claude review and a fix round. #1827 also had a live run with two participants on its branch: one reached its goal and declared reached, one could not and declared not_reached.

Not verified

  • A live Codex clock-end run with the new closing request, and a live blocked outcome; n=1 per outcome.
  • Whether asking for an outcome after a stop condition or dwell window changes how those summaries read.
  • The shared-world summary still counts a follower skipped at the handoff as using the app (#1831).
  • macOS and Windows.

Package

humanish@0.127.1: 1004 files, shasum 6b2d9569278dba2a7f9f0e635dce0086f52e83e9.