Skip to content

0.111.4: Encoded known values scrubbed, scripted traces scrubbed, a live command after a dry run

Choose a tag to compare

@danielgwilson danielgwilson released this 06 Oct 13:36
· 251 commits to main since this release
eb74165

humanish 0.111.4 scrubs a run's known values from more of its evidence and fixes three messages.
Every route now also finds a known value, such as a provider key or a subject.env value, in its
encoded forms: percent-encoded, JSON-escaped, base64, base64url and hex, and where a terminal color
code or an output chunk boundary splits it. A scripted run scrubs its known values from the step
trace in traces/<surface>.json and actor-<surface>.json, and from the app URL, scenario goal,
title and step labels it records. Two places keep a known value: a scenario file name, and the
path of a computer-use subject.appUrl. humanish review on a dry run names the command that
starts a live run, so next in review --json holds a string where it held null. A terminal
refusal for a missing runtime key no longer reads runtimeAuth "undefined", and a refusal that
names two signed-in coding agents reads "report".

This is a patch release. It removes and renames no command, flag, study field, bundle field or
library export. Four outputs change for the same input, which a script could notice:

# 0.111.3 0.111.4 What to do
1 humanish review --json on a dry run reported analysis.next: null, and review printed "Findings come from live runs." with no command after it. analyze show printed no command either. analysis.next is a string that starts a live run, such as npx humanish run try-live in a project init set up. review and analyze show end the findings line with "the next command runs try-live live." and print next: npx humanish run try-live. The command is run with the run's own study when its file sets mode: live, else run try-live when the project has that starter study, else run with the run's own study after a message to set mode: live in its file, else study list. analyze show --json on a dry run still reports analysis: null. (#1626) Read analysis.next as a string or null.
2 A live terminal study with no runtime key was refused with Live terminal-product studies declare runtimeAuth "undefined" and need CODEX_API_KEY or OPENAI_API_KEY in the environment (...) when the study left execution.runtimeAuth unset. Live terminal-product studies need CODEX_API_KEY or OPENAI_API_KEY in the environment, and neither is set., then one sentence on where the study's execution.runtimeAuth puts the key, then Pass one with --dotenv; the value is never persisted. The code HUMANISH_TERMINAL_RUNTIME_AUTH_MISSING and exit 2 are unchanged. (#1626) Match the refusal by error.code.
3 A computer-use or provisioned shared-world refusal for a missing OPENAI_API_KEY, on a machine where Codex and Claude Code both report a sign-in, read "Codex and Claude Code reports authenticated on this machine." "Codex and Claude Code report authenticated on this machine." The one-agent wording and the code are unchanged. (#1626) Match the refusal by error.code.
4 A known value written percent-encoded, JSON-escaped, base64 or hex stayed in run files on every route. A scripted run kept a known value as written in its step trace, app URL, scenario goal, title and step labels, and named a step and its screenshot from a label that held one. Each such span reads [REDACTED_SECRET], so a base64 run that encoded a known value is cut where the value was. A scripted step whose label or id holds a known value is recorded as step-NN by its position, with a screenshot named <surface>-step-NN.png, and a scenario id that holds one is recorded as scenario. (#1627) Read scripted steps by position where a label may hold a known value.

Changed

  • humanish review and humanish analyze show on a dry run name the command that starts a live
    run (row 1). (#1626)

Fixes

  • A live terminal study missing its runtime key is refused with a message that names
    CODEX_API_KEY and OPENAI_API_KEY and says where the study's execution.runtimeAuth puts the
    key (row 2). (#1626)
  • A refusal that names two signed-in coding agents reads "Codex and Claude Code report
    authenticated", where it read "reports" (row 3). (#1626)
  • A scripted run scrubs its known values (E2B_API_KEY, each subject.env value and the clone
    repository) from the step trace in traces/<surface>.json and actor-<surface>.json, and from
    the app URL, scenario goal, title and step labels it records. A step whose label or id holds a
    known value is recorded, and its screenshot named, as step-NN by its position, and a scenario
    id that holds one is recorded as scenario. Before, a known value in a goto path, the app URL,
    a step label or a page error stayed in those files and in run.json, review.md, events.ndjson and
    the Observer. (#1627)
  • Each route also scrubs a known value in its encoded forms: percent-encoded (as
    encodeURIComponent and encodeURI write it), JSON-escaped once or twice (also with non-ASCII
    characters as \u escapes), base64, base64url and hex. The base64, base64url and hex forms are
    matched from 8 characters. A value that a URL path encodes only in part, or that a terminal
    escape sequence splits (as written or JSON-escaped in a codex event), is found too, also across
    two terminal output chunks. The encoded span is replaced with [REDACTED_SECRET] and the rest of
    the text is kept as written. Before, only the value as written was scrubbed, so a value with a
    space, a quote or a backslash could stay in run.json, actor.json, the terminal transcript and the
    Observer. (#1627)

Known gaps

  • A known value in a scenario file name stays in the scenario path a run records.
  • A known value in the path of a computer-use subject.appUrl stays in the run files.

A study or persona id that holds a known value, and a value in an encoding the forms above do not
list, such as HTML entities or percent-encoding applied twice, are not checked.

Package

humanish 0.111.4 packs to 2.4 MB, 7.5 MB unpacked, 916 files (npm pack on release/0.111.4 at
823b68f), the same file list as 0.111.3. The package exports only ., with the same 13 values
and 31 types as 0.111.3 (pnpm api:proof). It needs Node 22.19 or newer, as before, and
@e2b/desktop 2.3.2 or newer as an optional peer.

The release tarball's shasum is 76e45f870a310f7f4085ca125293ea92e76d18de and its integrity
sha512-6tHAKPAM1mbdc0PDRdVG4lNXjfbDyKRKj8zmIX62jAGPN/ofGARbi65DlPSqRZfjxS5gxS0jMabSVdIHqsWyVA==.
npm reports the same shasum for humanish@0.111.4, with a provenance attestation (publish run
37469642738).

Checked before release

A live smoke on a tarball packed from main 79a5d60, in fresh projects under /tmp with a fresh
HOME and npm cache, DO_NOT_TRACK=1, keys passed to the CLI only through --dotenv, and
@e2b/desktop 2.4.0 beside humanish. The same studies ran on npm 0.111.3. Each run planted a known
value where its route records it, and every file of the run directory, with the command's own
output, was read for that value as written, percent-encoded, JSON-escaped, base64 and hex. The
value was a synthetic subject.env value with a space, quotes, a slash, an @ and an é on the
scripted, computer-use and shared-world routes, and the real OpenAI runtime key on the terminal
route, which has no subject.env. Files holding the value:

Route and plant 0.111.3 0.111.4
scripted clone of TodoMVC: a goto path, step label, goal and title 6 files: as written in review.md, percent-encoded in traces/desktop.json, JSON-escaped in traces/desktop.json, actor-desktop.json, run.json and both Observer files 0
computer use, openai-computer-use (try-live): the mission holds the value percent-encoded and base64 in a URL to type, and as written as a table name 5 files, percent-encoded and base64: run.json, the analysis, and three Observer files 0
shared world, 2 participants: one instruction holds the value as written, the other percent-encoded and base64 5 files, percent-encoded and base64, the same five 0
terminal, runtimeAuth: openai-env: the agent prints the runtime key lines as written, base64 on one line, and split by a color code 5 files, base64: actor.json, run.json, terminal-events.ndjson, terminal-transcript.txt and an Observer file; verify blocked 0; verify share_ready

On 0.111.3 the value as written was scrubbed on every route but the scripted one. Apart from that
terminal control, the raw OpenAI and E2B keys were in no run file, in any form. Priced spend with the dogfood run below:
about $3.8, $2.6 of it in automatic analysis.

  • computer-use, openai-computer-use: reached the goal in 8 turns, input growing every turn from
    3,020 to 14,970 tokens, cached input from turn 2. The participant typed the 26-character value
    and reported the table name; the run files show [REDACTED_SECRET] there.
  • shared world, 2 participants with caps.maxUsd: 1: both reached the goal, overlap proven; the
    per-participant cap warning printed.
  • scripted clone: 6 of 7 steps passed and the seventh, which looks for text the page does not
    show, failed as written; the step whose label held the value is recorded as step-02.
  • terminal: the agent ran the three commands and passed, share_ready. The transcript reads
    CODEX_API_KEY=[REDACTED_SECRET] in the plain and colored listings, and the base64 line reads
    Q09ERVhfQVBJX0tFWT1[REDACTED_SECRET]QpPUEVOQUlfQVBJX0tFWT1[REDACTED_SECRET]Qo=. A first version of
    the study asked for one command printing the key in named encodings; its agent ran no command
    and reported blocked on both versions, and again on main when the mission said what the run
    tests. A version that printed the whole environment as base64 recorded no key form, and verify
    blocked it for the sandbox's runtime paths inside the base64.
  • humanish review --json on a first-run dry run: analysis.next is npx humanish run try-live; 0.111.3 gave null.
  • After every run, reclaim --check exited 0 with clean, and verify exited 0 except on the
    two runs whose bundles it blocked, as said above. Raw sandbox ids appear only in
    sandbox-receipts.ndjson. E2B then listed 0 sandboxes tagged tool: humanish.

On the release commit 823b68f:

  • pnpm release:check exit 0 in 3.4 min: 449 test files passed and 10 skipped, 7,230 tests
    passed and 12 skipped; TUI 125 tests; lint 440 warnings at its cap of 440; api:proof "13 values
    and 31 types match"; public-surface scan of 2,834 text files and 523 binary assets; npm pack --dry-run shasum as above. pnpm docs:generate wrote 0.111.4 into cli.mdx.
  • pnpm release:dogfood: "release:dogfood ok", verdict blocked (blocked_approval), no-spend
    satisfied, $0.24. The participant ran the no-spend path and reported: "All 16 evidence checks
    passed; bundle status was share_ready." and "It produced no findings and recommends humanish run try-live." It stopped at the credentials the gate withholds. verify on the run:
    share_ready, 16 of 16; reclaim --check clean.
  • The post-publish check on the release tarball: 79 of 79 passed. Its 4 new rows cover the live
    command after a dry run, the two-agent wording, the terminal refusal (#1626), and a local
    scripted run whose goto path, step label, goal and title hold a synthetic E2B_API_KEY value,
    found in no run file in any of four forms (#1627). The same script on npm 0.111.3 passed 75 and
    failed exactly the 4 new rows. Check 20c now matches the dry-run findings line up to "Findings
    come from live runs", since 0.111.4 continues the line.
  • The consumer check: the 0.111.3 check's library and CLI consumer, plus two library steps and two
    CLI lines for this release. Typecheck 0 errors on both versions. Every output line is the same on
    npm 0.111.3 and the release tarball except the terminal refusal (library and CLI), review --json analysis.next (null, then npx humanish study list in a project without try-live), and
    an in-process computer-use participant whose mission and report quote a known value: 0.111.3
    left it percent-encoded in 7 run files and base64 in 3, and 0.111.4 in none.

Checked after publish

  • The post-publish check against npm 0.111.4, run with TMPDIR=/tmp: 85 of 85 passed, including
    the registry, latest and unpkg rows. A first run about ten minutes after publish failed at
    install because npm's CDN had not yet served the tarball (HTTP 404); it served at 13:33 UTC with
    the release tarball's shasum.

Not verified

  • macOS and Windows. Every check ran on Linux x64 with Node 24.12.
  • A terminal plant with a synthetic value: the terminal route scrubs only its runtime key and
    E2B_API_KEY, so the live check printed the real key. Its percent-encoded and JSON-escaped forms
    equal the key as written, so only the base64 and color-split forms differ from it.
  • A value split across two terminal output chunks, live: no command forced a chunk boundary inside
    a key.
  • The scripted base64 plant: the goto query that carried it is recorded as [redacted-query]
    on both versions, so the base64 form never reached a run file.
  • The local VM, email capture and receiving, local-agent participants and the TUI: not run on
    0.111.4.
  • A live run on the published package: the live smoke ran on a main tarball, and the post-publish
    rows are offline dry runs, refusals and a local scripted run.