fix: value-integrity — 5 backends corrupted secrets on read (v0.20.0)

[TechAlchemistX]

Summary

Five of fifteen backends silently corrupted secrets on read. Found from an external bug report against the macOS Keychain backend; that report was the visible tip of a wider, structurally different defect.

set() is unaffected on every backend, so no stored data is damaged and no migration is required. This is a read-path-only class.

The two mechanisms

1. macOS Keychain — undeclared hex encoding. security find-*-password -w hex-encodes its output, with no marker, whenever the value contains any byte outside printable ASCII. Confirmed triggers: newline, tab, 0x7F, and any non-ASCII UTF-8 — so the blast radius is not only PEMs but every accented or emoji-bearing passphrase.

get() now takes security's own declaration of the encoding. Shape decides only whether to ask; find-*-password -g decides what the value is. Encoding is never inferred from shape, because deadbeefcafe0123 is both an ordinary API key and valid hex.

Chose -w-then--g over always--g deliberately: -g prints the password to stderr, and this backend interpolates stderr into operation_failure_message. Making stderr the value channel would put plaintext one bad error path away from a log line.

2. vault / openbao / gcp / azure — trailing-newline truncation. All four ended get() with strip_suffix('\n'), correct only when the CLI appends a newline of its own. vault kv get -field=<f>, bao kv get -field=<f> and gcloud secrets versions access append nothing on a pipe; azure parses the value out of --output json, where no CLI newline exists at all. The strip ate the secret's own last byte.

PEMs, certificates and SSH keys are conventionally stored with a trailing newline and are rejected by openssl / ssh-keygen without it — so this broke exactly the values most likely to be multi-line.

The idiom is per-CLI, not globally wrong

Nine backends were verified clean and deliberately left alone: aws-ssm / aws-secrets (--output text), 1password (op read), conjur, keeper (--format=password), cf-kv (--text), doppler / infisical (--plain), local. Their CLIs do append a newline, so their strip is correct. Deleting the idiom globally would have broken all nine.

cf-kv was diagnosed as broken, fixed, committed — and reverted before shipping. An ad-hoc probe ran wrangler kv key get without the --text flag the backend actually passes. Without it wrangler appends nothing; with it, it appends. The probe "proved" corruption that does not exist, and the fix would have introduced truncation into a correct backend. Section 37 now carries a permanent cf-kv guard.

Why nothing caught this for 19 releases

Every assertion in the live matrix was grep-based, and grep cannot see a lost trailing newline or an undecoded hex payload. Six backends had a multiline unit test — all StrictMock, which returns whatever the test declares, so none could observe real CLI encoding.

Worse, the mock fixtures themselves encoded the wrong model: they fed "value\n", modelling these CLIs as appending a newline they never emit. That is what made the strip look correct under review. This PR corrects the fixtures rather than flipping the assertions, which broke 17 tests — the honest signal.

New permanent gate — smoke section 37

Byte-exact (cmp, not grep) round-trip across 14 of 15 backends, 104 assertions. Four values each: multi-line PEM, non-ASCII UTF-8, legitimate trailing newline, and a hex-shaped secret (deadbeefcafe0123) that fails if anyone reintroduces shape-inferred decoding.

Each value is seeded with the backend's native CLI and read back twice — natively and through secretenv. That split is load-bearing: it separates "this backend cannot hold these bytes" (SKIP, a documented limitation) from "SecretEnv corrupts them" (FAIL, a defect). Without it, Doppler's multi-line rejection and 1Password's write-time newline truncation both read as SecretEnv bugs.

Verification

• Live smoke: 916/916 PASS, 0 FAIL across all 15 backends (10 SKIPs, all documented limitations).
• 1136 workspace tests, 0 failures.
• Stable-channel cargo fmt --check; CI-form clippy --all-targets --workspace.
• Regression proven both directions: simulating the pre-fix code makes the PEM / UTF-8 / trailing-newline tests fail, while the hex-valued and quotes guards pass on both versions — the correct signature.
• Version bump touched root Cargo.toml (23 pins) and crates/secretenv-mcp/Cargo.toml's hardcoded core pin; all 24 crates resolve at 0.20.0.
• release.yml audited — no change needed; it publishes via cargo publish --workspace, so the dep-order incident class is structurally gone.

Known non-defects (recorded, not fixed)

• Doppler rejects multi-line values at the API.
• 1Password truncates a trailing newline at write time — op item create stores abc given abc\n. SecretEnv reads back faithfully what 1Password holds.
• Keeper is not covered by section 37 — seeding needs record-add with a record type and folder, which would mutate the operator's real vault. Both read paths verified instead (JSON path applies no strip; --format=password measured live to append a newline). Recorded as an explicit SKIP so the gap stays visible.

Reviewer note

Section 37 is ~470 lines of new harness code and produced four defects of its own during construction (perl @VAL array interpolation, exact-match arg substitution, an unconditional pre-trim that destroyed the evidence, and Doppler fixture pollution that broke a later run). None reached the product, and all were caught by the two-sided comparison — but a green run proves the backends behave, not that the gate is correct. It's the part of this changeset most worth a careful read.

🤖 Generated with Claude Code