v0.21.0 — every secret looked like every other

Two fixes, both about secrets written into a test, and one of them a wrong PASS. Both change what a run or a draft does rather than only what it prints, which is why this is a minor.

Every secret looked like every other

Since the mask existed, every registered {{env.*}} value was redacted to the same ***. That kept values out of prompts and reports, which is its job. It also erased the one thing a judgment on a secret needs: which secret it was.

The judge sees the step through the mask, the same as the page. So a step verifying text equal to a secret reached it as "***", the page showed "***", and the judge was right to call them equal, whatever the page actually said. #87 was filed as the mild half of this, "such a step can never pass". Measured, it passes, and it also passes when it's false:

step verifies	page shows	before	now
Unknown promo code "HUNTER2".	Unknown promo code "HUNTER2".	3/3 PASS	3/3 PASS
Unknown promo code "SAVE99".	Unknown promo code "HUNTER2".	3/3 PASS	3/3 FAIL

Both HUNTER2 and SAVE99 were registered secrets. For a tool whose output gates merges, a deterministic PASS on a wrong page is the worst thing it can say.

A redaction now names its variable:

- status: Unknown promo code "[redacted PROBE_SECRET]".

The label carries the name and nothing derived from the value: no length, no prefix, no hash, because each of those narrows the value and a hash of a short password is a lookup. The same secret always gets the same label, whichever form of it matched (an uppercased echo, a percent-encoded URL), and two secrets get two. After the change the judge explains its FAIL itself: "these are two different redacted values".

The same change settles the opposite complaint. A value put in {{env.*}} for portability rather than secrecy (a tenant, a test account) used to be unassertable. It is now asserted on by identity, without being unmasked.

One door had to close with it. The executor only admits a typed value that came from the step or from the page the model read, and the page it read is the masked one. A label is on it, so without a guard the model could copy [redacted TEST_EMAIL] into a field and have it typed literally. A fill or select containing a label is now refused, with an explanation the model can act on.

Output change: logs, JUnit and the HTML report show [redacted NAME] where they showed ***. Anything that matched *** in those files needs to match [redacted instead.

"Never write a password, token or key"

That was the planner's rule, and an email address is none of the three. So a login draft used {{env.TEST_PASSWORD}} for the password and a literal address for the account, and it looked right to the person reviewing it, because the line that mattered most was handled.

The rule now names the property: any value that identifies an account or a person, with email, username and account number as examples, not as the list. On the demo's login page, which prints its own credentials as a hint:

prompt	drafts using a placeholder
old	0 of 5
new	4 of 5

With the old prompt every draft copied the hint, password included, which the old rule already forbade. That led to the second half of this fix. The planner's secret check only fired on a quoted value, and a model doesn't quote: all five of those drafts wrote Fill the Password textbox with demo123 and were accepted. A step naming a credential now counts as carrying a value when it's quoted, or when it enters a named value by the definition run's authoring check already uses. Replayed offline, all six literal-password drafts from the measurement are refused, and the four with placeholders aren't.

plan also warns when a draft writes an email address the page it was drafted from doesn't show. The draft is still written: a literal address is a wrong test, not a leak, and the reviewer is who should fix it. An address the page does show isn't reported, since verifying a displayed contact is legitimate. That means a page printing its own credentials hides a copied one, and the authoring reference says so rather than implying otherwise.

Verified before tagging

• The #87 experiment was rerun unchanged after the fix: true 3/3 PASS, false 3/3 FAIL, and neither value appears in any log.
• Mutations on both fixes, each caught by at least one test. One of them first caught a hole in the tests themselves: removing the quoted-value branch turned nothing red, because the only quoted test was also an entry step. A test now pins a quoted secret in a step that enters nothing.
• The Dogfood workflow on the merge base, in CI: 8 of 8, Score 100 against --min-score 80, 93 model calls. Worth saying plainly: that suite uses no {{env.*}} value, so it shows the prompt changes didn't regress the common path. It doesn't validate the label. The experiment above does.

Also

#112, the wrong PASS filed during 0.20.0, was measured 20 times after the demo's CSS fix and didn't reproduce. It's closed as not reproduced, with the data on the issue.