Skip to content

v8.11.0 — a credential report that does not gate

Choose a tag to compare

@Zandereins Zandereins released this 10 Aug 07:05
· 446 commits to 7d37126ee988b4ad127410ae7b2f32a0d53afbf9 since this release

Three imports from microsoft/SkillOpt (MIT). No score changes, no exit-code changes — every pinned --min-score behaves exactly as in 8.10.1.

Added

Credential findings in score, score --json, doctor, verify and the GitHub Action. Schliff points out strings shaped like an AWS access key, a GitHub token, an Anthropic or OpenAI key, a Slack token or a Google key. A finding carries the vendor and the line number, never the value.

Nothing gates on it. A green pipeline stays green: verify still exits on --min-score and --regression alone, and the Action annotates with a warning without failing the job.

The limitation, stated plainly: a token's shape does not tell a live key from a documentation example. Placeholders that name themselves are excluded — AWS's own AKIAIOSFODNN7EXAMPLE, <your-key>, ${VAR} — but a placeholder that merely looks real will be reported, and a real key in a shape schliff does not know will not be. Measured over ~740 real files, every finding the scan produced was documentation about credentials. JWTs are deliberately not detected: the jwt.io sample and Supabase's anon key are public by design and structurally identical to a service key.

This is a report, not a secret scanner. If you need enforcement, run a dedicated one — or read the credentials field of schliff score --json and fail your own build on it.

Fixed

  • The improvement loop no longer applies a gradient to the wrong file. A gradient naming eval-suite.json as its target is not written to SKILL.md.
  • Train/val split for the loop. Edits derive from train, the keep/revert gate judges val, and holdout_leaked says when the two are not disjoint. Every population the gate cannot see falls back to the full suite, so a patch that destroys quality or edges can no longer be kept unseen.
  • The loop's stop condition uses the score you can reproduce. composite >= 98 is evaluated on the same basis schliff score reports.
  • Redaction gaps: all five GitHub token classes, both ODBC password spellings including short values, and modern sk-proj- keys whose body carries hyphens. $PWD and lowercase pwd= still survive.

Security

  • The Action refuses paths that resolve outside the workspace. Both skill-path and eval-suite are checked before schliff reads anything.

Decisions

ADRs 0010–0020 record the import. Two are worth reading if you care why the credential check reports rather than gates: 0019 (nothing gates) and 0020 (shape does not separate a credential from a placeholder).

Full changelog: v8.10.1...v8.11.0