Repository navigation
0.130.0: Analysis reads traces up to 32 MiB one at a time and names what it refused; the bundle estimate counts the mission
humanish 0.130.0 lets analysis read long runs that 0.129.0 refused. Analysis reads each
participant's trace up to 32 MiB, as verify and the Observer do, one trace at a time, and keeps of
each only what selection can use. humanish analyze names the file it refused and why. The bundle
estimate counts the study's mission in run.json, so its run.json warning can fire. This is a minor
release because a run.json past 16 MiB is now refused as ANALYSIS_SOURCE_TOO_LARGE where it was
ANALYSIS_SOURCE_UNAVAILABLE.
Changed
- The types for run.json as a reader parses it, and for the Observer's data, leave out each participant's per-request lists (
tokenUsage.turns,conversation.requestsandproviderRequests), which a v4 run.json holds only in the trace file.isRunBundle, the trace loader's input and the readers that cast run.json useStoredRunBundle, whose streams'actorandliveActorareWithoutRequestLists<...>, andObserverStreamextendsStoredRunStream.pnpm typechecknow refuses a read of a list there; the full actor trace type, withconversation.requestsrequired, let one compile and throw on a v4 file.RunBundle, whichloadRunBundlereturns with the lists joined, is unchanged. (#1835, #1853)
Fixes
-
The bundle estimate counts the study's mission in run.json. run.json holds it once for the run, once in each participant's stream and once more in the feedback candidate of each participant that reports friction or stops, and the estimate counts that last copy for every participant. It counted run.json at 4,400 bytes plus 16,500 a participant whatever the mission, so its warning that run.json passes the 16 MiB analysis reads could not fire within the 100-participant cap. 100 participants with a mission of about 75 KB now get that warning from
study check,humanish runanddoctor --study, with advice to shortenactor.missionor split the participants across studies. On dry runs, 20,000 more bytes of mission added 40,000 bytes to run.json with 1 participant and 100,000 with 4. The Observer page estimate counts the mission once for the run and once for each participant, since the page holds no feedback candidate. (#1836, #1847) -
The Observer keeps a viewer's saved moments on reload when one is at frame 100,000 or later. It refused such a moment, and with it every saved moment in the browser, up to 50, so the list came back empty. A saved frame may now be any whole number from 0 to
Number.MAX_SAFE_INTEGER, the range analysis accepts. A participant reaches frame 100,000 after about 278 hours at a model turn every 10 seconds. (#1842, #1847) -
Analysis reads each participant's trace up to 32 MiB, as the trace loader, verify and the Observer read it. It refused a run with a trace past 16 MiB (
ANALYSIS_SOURCE_TOO_LARGE), which a participant reaches after about 3,570 model turns at the 4,700 bytes a turn the bundle estimate counts. Evidence capture now reads the traces one at a time and keeps of each only the entries selection can admit, where it held every trace at once. On 25 synthetic participants with 15.3 MiB traces, capture raised peak memory by 987 to 990 MiB on 0.129.0 and by 240 to 282 MiB now, and both gave the same analysis input. With 29.6 MiB traces, which 0.129.0 refused, capture alone raised it by 327 to 375 MiB for 25 participants and by 284 to 321 MiB for 5. run.json is still read only up to 16 MiB. verify, which analysis runs first, still holds every trace at once and scans the run as one string, so on a run whose traces together pass about 512 MiB, such as 25 participants at 30 MiB, verify stops withRangeError: Invalid string lengthand analysis withANALYSIS_UNAVAILABLE. The bundle estimate warns about a trace past 32 MiB, where it warned past 16 MiB. (#1840, #1854) -
humanish analyzenow says which file of a run it refused and why. A run.json or participant trace without the fields analysis reads (ANALYSIS_SOURCE_INVALID) printed "Analysis could not be completed safely. Source evidence was not changed."; the message now names the file and the problem, such as a trace that holds two items with one id. A run.json past 16 MiB was refused asANALYSIS_SOURCE_UNAVAILABLE, "The source recording is missing, unsafe, or exceeds the input limit."; it is nowANALYSIS_SOURCE_TOO_LARGE, and the message gives the file's size and the 16,777,216-byte limit and says to split the participants across studies. Both messages end "No request was sent."ANALYSIS_SOURCE_UNAVAILABLEnow reads "The source recording is missing or unsafe." A trace past 32 MiB, or one that does not match its index, still fails verify, which analyze runs first, and analyze reportsANALYSIS_VERIFY_FAILED. (#1841, #1854)
Checked before release
- CI on this PR is the gate of record: the sharded gate on both Node versions, site, Observer proofs, TUI, guest desktop and secret scan.
pnpm gateon the release commit passed. - The post-publish check rehearsed on the release tarball (
pnpm packed:check): all 125 rows passed. The registry and unpkg rows run after publish. pnpm release:dogfood: a participant installed the tarball and ran the offline first-run study (share_ready, 16 of 16 checks, $0). It then stopped at the credentials the gate withholds.- Live on the release tarball (hosted E2B, Chrome, keys from the user store):
- try-live with analysis: the run passed, and analysis wrote 3 findings and 3 design findings. verify passed.
humanish analyze --dry-run --max-cost 3 --jsonon that run: ok. With run.json padded to 16,777,217 bytes it answeredANALYSIS_SOURCE_TOO_LARGE, naming the size and the 16 MiB limit and ending "No request was sent." After the file was restored, verify passed.study checkon try-live with 100 participants and an 80,000-byte mission: run.json is estimated at about 16.9 MiB, with the warning and the advice to shortenactor.mission. On 0.129.0 the same file estimated 1.6 MiB with no warning.
- Before merge:
- Each change had a Claude review and a fix round.
- The analysis change (#1854) had two Codex reviews. Round 1 found three issues, each fixed with a test that failed first. In round 2, 508 synthetic inputs that 0.129.0 accepted gave identical analysis input.
- The type change (#1853) had a Codex review that found no runtime change.
- Peak memory added by evidence capture, 25 synthetic participants with 15.3 MiB traces: +987 to +990 MiB on 0.129.0, +240 to +282 MiB on this release.
Not verified
- Analysis of a live trace between 16 and 32 MiB. Unit tests and the synthetic memory measurements cover it.
- Memory through
humanish analyzeend to end. The figures above are for one evidence capture call. - A live run with a long mission, and saved Observer moments past frame 99,999 in a real browser.
- verify on runs whose traces together pass about 512 MiB. It throws
RangeErrorthere (#1848). - macOS and Windows.
Package
humanish@0.130.0: 1006 files, shasum 0405653cb192100d55d9e58a0956c2687b144564.
🤖 Generated with Claude Code