Skip to content

opena2a-cli 0.10.12

Latest

Choose a tag to compare

@github-actions github-actions released this 30 Jul 00:34

Published via npm Trusted Publishing (OIDC) from .github/workflows/release.yml. No long-lived token is involved. Verify the supply chain yourself:

npm view opena2a-cli dist.attestations --json
# predicateType: https://slsa.dev/provenance/v1

Security

  • Delegated tools now receive an allowlisted environment declared by their own adapter registry entry, instead of the operator's entire environment (#228). Measured on one developer machine: 128 variables reached the child process, 11 of them credentials; now 20 and none. The contract lives per ADAPTER_REGISTRY entry because these adapters run first-party tools with real environment contracts — secretless-ai needs VAULT_*/AWS_* because it is a credential broker, and hackmyagent needs NANOMIND_URL/ANTHROPIC_API_KEY or it silently returns different findings with a zero exit code. Availability probes get a narrower environment with no credentials at all. Scope note: this reduces environment-borne exposure only. HOME is forwarded by design, so ~/.npmrc and ~/.aws/credentials remain reachable by any child, and the guard matches variable names, not values.
  • shield recover --verify verifies before it recovers, which is what the flag always claimed (#228).

Fixed

  • benchmark no longer certifies controls it never evaluated (#250). This is the headline behavior change. A control counted as passing whenever no finding mentioned its check id, and the only source consulted evaluates none of the 39 OASB L1 controls — so every Certified this command ever printed meant "the scanner did not mention these ids". Concretely, on a directory containing a hardcoded Anthropic key:

    0.10.11 0.10.12
    Rating Certified Not Passing
    L1 100% (39/39 checks) 67% (3 of 39 controls evaluated)

    Scoring is now coverage-gated: unevaluated controls leave both sides of the ratio and are disclosed on every run. See Known issues for what that means for Certified today.

  • Every score line states its scope (#252). init, scan, review and benchmark measure different things, and four bare numbers for one directory read as a contradiction.

  • init detects credential-shaped values in template env files (#227).

  • review's verdict line no longer disagrees with the composite band it reports (#221).

  • Telemetry is suppressed in CI and under DO_NOT_TRACK (#240); the disclosure line reads off in those environments rather than claiming on.

Changed

  • hackmyagent pin 0.23.6 -> 0.25.1. 0.10.11 shipped 0.23.6, so the delegated scanner moves three steps for anyone upgrading and aggregate scores move with it (measured on the empty-directory baseline: 0.23.6 = 98, 0.25.1 = 96). Finding sets are byte-identical across the last step on every affected fixture, so the movement is 0.25.1 removing score inflation rather than losing detection. 0.25.1 also carries the artifact-intent reconciliation that stops scan printing a raw classifier label on a clean artifact.

Known issues

  • No project can reach Certified on benchmark today, and anything that previously read Certified will now read Partial. That is #250 working, not a new failure: Certified requires complete coverage of the level and the consulted scanners evaluate 3 of the 39 L1 controls. Every run says so explicitly (36 of 39 controls were not evaluated, these are NOT counted as passing). Widening coverage needs the semantic pass wired in; until then a thin true number is reported instead of a complete-looking false one.
  • Following harden-soul still costs 5 points on init (#251, reopened). One line of the generated governance names the phrases it resists, and the matcher counts that as an override pattern: init reads 75 before and 70 after. Two context-aware matchers were attempted and both were reverted — one was defeated by a single apostrophe in prose, the other by two apostrophes or by quoting the payload, each losing a true positive the plain matcher caught. The plain matcher is restored and every evasion is pinned as a regression test. The label half of #251 is fixed by the pin above.
  • opena2a scan still reports 96/100 on a file with a hardcoded OpenAI key; the delegated scanner marks CRED-002 as passing there (hackmyagent#316). init, protect, review and benchmark all flag it, so scan is the outlier.

Full Changelog: v0.10.11...cli-v0.10.12