Fail closed on keyed skill upgrade verdicts - #6628
Conversation
thv skill upgrade skipped the signer guard entirely whenever --allow-signer-change was passed, so the project-wide override dropped a still-valid key pin on every key-pinned skill, and a transient verifier fault during planning was enough to re-anchor an artifact that still carried a good signature by the pinned key. The keyed guard also called an unsigned candidate a signer change, sending the user to a flag that cannot help since upgrade has no unsigned-consent flag, and diagnosed a mid-migration artifact as a damaged signature without probing whether it had moved to keyless. Mirror the verdict design already reviewed for thv ai-plugin upgrade: measure the candidate against the pin once, treat only a conclusive mismatch plus a verifying keyless signature as a move, fail everything undecided in both modes, and narrow the override per entry so needing it for one skill does not unpin the rest. Remediation text now names a runnable project-scoped reinstall. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Samuele Verzi <samu@stacklok.com>
6191b51 to
217bc7d
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #6628 +/- ##
=======================================
Coverage 78.89% 78.90%
=======================================
Files 778 778
Lines 77598 77657 +59
=======================================
+ Hits 61223 61274 +51
- Misses 16370 16378 +8
Partials 5 5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
JAORMX
left a comment
There was a problem hiding this comment.
Panel review completed against 217bc7dacc0336c377f2cec0b129a7b7f64545c1.
No merge blockers found. The keyed-candidate verdict separates conclusive key migration from unsigned, invalid, and operational-failure cases; --allow-signer-change permits only the verified key-to-keyless transition. The narrowed per-entry override preserves unrelated pins, and the regression coverage exercises the relevant verification and lock-preservation paths.
Exact-head CI is green.
Summary
thv skill upgradeandthv ai-plugin upgradeare meant to be the same operation over two lock files, and the plugin side went through three review rounds on #6525 that fixed how a key-pinned entry is judged. None of those fixes reachedpkg/skills/skillsvc/upgrade.go, sothv skill upgradestill has every one of the bugs the review found:--allow-signer-changeis passed, the signer guard is skipped entirely (planUpgradegated it on!opts.AllowSignerChange). The project-wide flag then unpins every key-pinned skill, including ones whose candidate still verifies against the pinned key, and a transient verifier fault during planning is enough to re-anchor an artifact that still carries a good signature by that key. This is the fail-open the Honour a pinned cosign key on plugin sync and upgrade #6525 review called critical.signer-change-blocked, so the CLI told the user to pass--allow-signer-change. Upgrade has no unsigned-consent flag and that override is not one: it re-verifies from scratch, which an unsigned artifact still fails.VerifyOCIWithKeyreturnsErrKeylessSignedonly when every bundle is keyless, so an artifact mid-migration with a stale key-pair bundle beside a valid keyless one came back asErrSignatureInvalidand was reported as a different key.--public-keywithout the command, the skill argument, or--scope project, andthv skill installdefaults to user scope where--public-keyis rejected.This PR mirrors the plugin verdict design into the skills service, function for function, so both halves are behaviourally identical and reviewable against code already approved:
judgeKeyedCandidatemeasures the candidate against the pin once and returns one of three verdicts. Only a conclusive keyed mismatch (ErrKeylessSignedorErrSignatureInvalid) followed by a keyless signature that verifies is a genuine move. An operational error stops before the keyless probe and is never read as evidence about the key.recordKeyedVerdictis the only mode-dependent step: a genuine move blocks without the override and proceeds with it. Undecided candidates fail in both modes, so the override can never turn a failed measurement into permission to re-anchor.resolveSignerPolicynarrows the project-wide--allow-signer-changeto the entry it applies to, andapplyUpgradepasses that narrowed flag to the install. A skill that needs the override no longer unpins the key-pinned skills beside it.unsigned-rejectedfailure; the mixed-artifact case is a move; different key names the runnablethv skill uninstall … --scope projectthenthv skill install … --scope project --public-key <path>sequence.The keyless guard's own unsigned arm (
guardSignerChange, stillsigner-change-blockedfor an unsigned candidate) has the same dead-end shape on both services and is deliberately left for #6629, which stacks on this one and needsprojectReinstallCommandfrom here.Type of change
Test plan
task test)task test-e2e)task lint-fix)task docs) — no CLI or API surface changedCoverage mirrors the plugin side:
TestJudgeKeyedCandidate(seven verdict cases, each run throughrecordKeyedVerdictin both modes, asserting the probe is not consulted on an operational fault),TestJudgeKeyedCandidate_UndecodablePinnedKey, and four integration tests over a real install → upgrade against a real OCI store: pin verified on both the guard and the install, genuine key-to-keyless move re-anchors, same-key candidate keeps its pin under the override, and an operational fault under the override leaves the lock byte-identical.TestPrintUpgradeResultSignerRenderingpins the three CLI renderings the design depends on.Manual mutation check: reverting
applyUpgradeto passopts.AllowSignerChangeinstead of the narrowed flag makesTestUpgrade_AllowSignerChangeKeepsSameKeyPinfail, so the regression coverage is live rather than incidental.Changes
pkg/skills/skillsvc/upgrade.gojudgeKeyedCandidate,recordKeyedVerdict,resolveSignerPolicy,conclusiveKeyedMismatch,keyedFailureReason,keyedFailureMessage,projectReinstallCommand);upgradePlan.allowSignerChange;guardKeyedSignerChangeremovedpkg/skills/skillsvc/upgrade_verify_test.goTestGuardKeyedSignerChangereplaced byTestJudgeKeyedCandidate;keyPinnedUpgradeFixtureand four end-to-end keyed upgrade testscmd/thv/app/skill_upgrade_test.goTestPrintUpgradeResultSignerRenderingdocs/arch/12-skills-system.mdupgradebullet rewritten to describe the verdict semantics and the per-entry overrideDoes this introduce a user-facing change?
Yes, for
thv skill upgradeon key-pinned entries:--allow-signer-changeno longer drops a key pin whose candidate still verifies against it, and no longer drops one on a verifier fault. Previously both silently re-anchored to keyless.failed [unsigned-rejected]with a runnable reinstall command, instead ofsigner change blockedpointing at a flag that cannot resolve it.Special notes for reviewers
pkg/plugins/pluginsvc/upgrade.goafter Honour a pinned cosign key on plugin sync and upgrade #6525, not a redesign. Reading the two files side by side should show only theskills/pluginstype prefixes, thenewRef, newDigestvsresolvedLatestparameter shape, and thethv skillvsthv ai-plugincommand names differing.guardSignerChange's keyless unsigned arm is unchanged here on purpose and is Reject unsigned upgrade candidates in both modes #6629. Writing that PR's override-mode test surfaced that onmainthv skill upgrade --allow-signer-changesilently converts a signer-pinned skill tounsigned: truewhen the candidate is unsigned; see Reject unsigned upgrade candidates in both modes #6629 for the mechanism.🤖 Generated with Claude Code