feat: add repo-setup audit skill by tonyandrewmeyer · Pull Request #22 · canonical/charm-tech

tonyandrewmeyer · 2026-07-02T01:14:51Z

This PR adds a skill that summarises and automates (deterministically where possible) the repository setup (including SSDLC) work I've done this cycle.

The skill:

Is tier-aware (product / canonical / personal), agent-generic (not Claude-specific).
Audits a repository against the 26.10 baseline: SSDLC compliance, supply-chain hygiene, best-of-class extras.
Ships deterministic check & fix scripts plus consolidated references (SSDLC framework, settled carve-out decisions, tools measured and skipped, sweep history, open investigations).

I've tested this (with Claude Code) against a Canonical Charm Tech repo in the "product" group, a non-product Charm Tech repo, and a personal repo that might end up a Canonical repo some day.

Tier-aware (product / canonical / personal), agent-generic skill that audits a Canonical Charm Tech repository against the 26.10 repo-setup baseline: SSDLC compliance, supply-chain hygiene, and best-of-class extras. Ships deterministic check + fix scripts plus consolidated references covering the SSDLC framework, settled carve-out decisions, tools we measured and skipped, sweep history, and open investigations. Lifted from the canonical-work-queue staging tree where it was iterated during the 26.10 repo-setup work.

tonyandrewmeyer · 2026-07-02T02:30:23Z

+<!--
+For detailed dev-environment setup, build, and test instructions, link here to
+the substantive doc if one exists (HACKING.md, docs/contributing.md, etc.).
+Most Charm Tech repos keep this section inline rather than redirecting.
+-->
+


Suggested change

Co-authored-by: Tony Meyer <tony.meyer@gmail.com>

…cycle The Canonical Security "Repository security" and "How-To: Secure a repo" pages published 2026-07-01 recommend Require signed commits on protected branches. Team decision 2026-07-02: not required this cycle; re-evaluate in 26.10+1. Rationale in decisions.md. Also silences a would-be gap-report entry: a repo without require_signed_commits on its default-branch ruleset is not flagged in this cycle.

… when membership > Charm Tech team Charm Tech maintainer scope. A CODEOWNERS file that maps everything to the same team that already required-reviews every PR adds friction without filtering signal. CODEOWNERS is therefore only required for a charm-tech repo when its maintainer/contributor set is broader than the Charm Tech team — cross-team ownership, external contributors, or per-directory owners mapping to different sub-teams. In 26.10 that's charmlibs only. A check should not flag "no CODEOWNERS" on a Charm-Tech-only repo, and should flag a CODEOWNERS file that maps everything to the Charm Tech team wholesale.

Adds a per-repo secret-handling audit modelled on the Canonical Security "Repository security" reference (Secrets section) and the 2026-07-02 fleet audit recorded in roadmap/26.10/repo-setup/security-docs-gap.md rows #32-#36. Detects, in .github/workflows/*.y*ml: * workflow-level env: with ${{ secrets.* }} * job-level env: with ${{ secrets.* }} Both over-scope: every job or step in reach can read the secret. Step-level env: is the correct scope. * run: lines that echo / printf / cat a secret expression. The reference page warns "log masking is not guaranteed for every transformation" — keep secrets out of shell-visible strings. * secrets: inherit on reusable-workflow calls. Pass named secrets so the callee's surface is auditable. Env-scope checks need python3+PyYAML for reliable scope resolution; textual checks (echo/inherit) run without either. Verified against: * canonical/charmhub-listing-review @ main → fail (1 echo hit on review-request.yaml:31 — same finding as the fleet audit) * canonical/charmhub-listing-review @ security/drop-redundant-gh-auth-echo → pass * canonical/charmlibs → pass

…config Policy change: every third-party GitHub Action pins to a SHA — no allowlist exceptions for actions/, github/, pypa/, canonical/. The pypa/* open investigation resolved into the uniform posture rather than a per-org carve-out. .github/zizmor.yaml is no longer needed (the default unpinned-uses rule enforces the policy), so the config template and its fix script are removed. The zizmor-config check now only requires that a workflow invokes zizmor. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…ss-stale not required The Canonical Security "How-To: Secure a repo" page recommends both Require conversation resolution and dismiss stale reviews on new commits on protected branches. Team decision 2026-07-02: not required this cycle. Conversation resolution required: the team already treats unresolved threads as a review-blocker socially; a merge gate adds friction (chase every "nit" the reviewer intended as advisory) and every follow-up commit re-triggers the resolution round. Net-negative for Charm-Tech-sized PRs with a small consistent reviewer set. Dismiss stale reviews on new commits: most Charm Tech PRs iterate quickly; auto-dismissing on every push forces a full re-review even for a rebase or typo fix. The Require last push approval flag (already in the CRA baseline) covers the substantive risk without that churn. Both re-evaluated in a future cycle if the team scales or an incident ties to their absence.

…refix Match the resolved rule from the dependabot-conventions spec (canonical#17): 'chore:' with no scope, on both the pip and github-actions ecosystems. Setting only 'prefix: chore' produces 'chore: bump ...' automatically — Dependabot inserts the colon-space when the prefix ends in a letter — so 'include: scope' is unnecessary and would produce the non-conforming 'chore(pip): ...' / 'ci(github-actions): ...' shape. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Adds a per-repo check for the "Lockfile(s)" section of the Canonical Security "How-To: Secure a repo" page: every uv run / uv sync invocation in CI must pass --locked (or --frozen) so that a stale uv.lock fails the build rather than silently re-resolving. Prefers --locked over --frozen. --locked asserts uv.lock is fresh vs pyproject.toml and fails otherwise, forcing the PR author to commit a regenerated lockfile — the resolution delta becomes reviewable. --frozen skips the freshness check entirely, so drift is silently masked. Scope: * Reads .github/workflows/*.y*ml AND the top-level Makefile (api_demo_server calls uv run through `make lint` / `make integration` — the CI-shell-out path needs to be covered). * Classifies each invocation: locked / frozen / bare / skipped-*. * Skips patterns that don't touch the project lockfile: uv run --no-project --script / --with-requirements, uvx, and uv tool install/run. * na when there's no pyproject.toml / uv.lock (Go repos etc.) or no project-scoped invocations found. Verified 2026-07-02 against: * canonical/jubilant → fail (9 bare in Makefile) * canonical/charmhub-listing-review → fail (3 frozen, 2 bare — the "frozen exemplar" designation was overstated; two bare uv run invocations remain) * canonical/pebble → na (Go, no uv) * canonical/api_demo_server → fail (6 bare across security.yaml and Makefile) Matches the pattern of the workflow-secrets check: bash script that shells out to python3 for line-level classification, emits JSON per scripts/lib/common.sh conventions.

Adds a per-repo check that pyproject.toml sets [tool.uv].exclude-newer to a rolling ≥7-day quarantine. Rationale (Canonical Security "How-To: Secure a repo" — Minimum release age section): Dependabot cooldown protects PRs Dependabot opens; exclude-newer protects every OTHER dep-resolution path (manual uv add, uv lock regens, uvx bootstraps, CI re-resolves). Per uv docs, exclude-newer accepts three formats: - Friendly duration (rolling window): "7 days", "1 week", "30 days" - ISO 8601 duration (rolling window): "P7D", "P30D", "PT24H" - RFC 3339 timestamp (absolute snapshot): 2026-01-01T00:00:00Z The check accepts all three but only friendly-duration and ISO 8601 values contribute to the ≥7-day computation. Snapshots pass with a note recommending a rolling value instead — snapshots drift silently as time passes and need periodic manual bumping. Verified 2026-07-02 against: * canonical/jubilant → fail (uv.lock but no [tool.uv]) * canonical/pytest-jubilant → fail (same) * canonical/api_demo_server → fail (same) * canonical/charmlibs → fail ([tool.uv] present, exclude-newer missing) * canonical/pebble → na (Go, no pyproject.toml) * canonical/charm-ubuntu → na (no pyproject.toml) * synthesised "7 days" / "P1W" → pass * synthesised "3 days" → fail (below threshold) * synthesised RFC 3339 timestamp → pass with snapshot note Refinement: a project with uv.lock but no [tool.uv] IS a uv project that just hasn't declared the config section — that's a fail (add the section), not na. The na verdict is reserved for projects with no uv.lock and no [tool.uv], where uv might not even be in use.

…v exclude-newer Records the finding that Go's module resolver has no native minimum-release-age concept — no go.mod directive, GOFLAGS value, or env var equivalent to `[tool.uv].exclude-newer = "7 days"`. The residual vector (a dev running `go get -u <mod>@latest` or `go mod tidy` locally, pulling a fresh release into go.sum before Dependabot cools it down) is unaddressed on pebble and concierge. Enumerates four workaround options (custom check, private proxy, PR-diff inspection, accept-and-monitor) and defers to accept-and- monitor for the 26.10 cycle. If the fleet's Go footprint grows or an incident traces to this vector, prefer PR-diff inspection. Explicitly notes: a check should not flag pebble or concierge for missing a Go-side release-age control in the 26.10 cycle.

Converts the skill's 37 shell scripts to Python for reviewability and maintenance. All scripts are stdlib-only except dependabot.py and workflow-secrets.py, which use a PEP 723 uv run shebang to pull in PyYAML (the shell versions already had PyYAML as a soft optional dep via embedded python heredocs). Preserves check ids, summary strings, evidence keys, remediation shapes, and exit codes (0/1/2/3 = pass/fail/na/unknown) so the JSON report contract is unchanged. Umbrella runner verified end-to-end against this repo (27 checks emit valid single-line JSON, aggregate report matches SKILL.md schema). Also fixes a latent bug in add-dependabot: the shell version referenced assets/dependabot.yml.template but the asset is dependabot.yaml.template. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Adds a per-repo check that [tool.uv].no-build = true is set in pyproject.toml. uv refuses any sdist install and only accepts wheels, closing the setup.py / PEP 517-hook arbitrary-code-execution vector at install time. Rationale (Canonical Security "How-To: Secure a repo" — Install scripts): source distributions run setup.py / build hooks during install; wheels don't. Refusing sdist eliminates a well-known supply-chain vector at zero cost for typical Charm Tech repos. A fleet-wide scan (2026-07-02) found 0/571 dependencies across every uv project were sdist-only, so this is a zero-cost policy to adopt. Per-package escape hatches via no-build-package = ["some-pkg"] are recognised and reported in evidence for future one-off exceptions. Written in Python matching the ported skill layout (was initially drafted in bash, then rewritten to match the .py convention after the skill was ported). Verified 2026-07-02 against: * canonical/jubilant → fail (uv.lock present but no [tool.uv]) * canonical/pebble → na (Go, no pyproject.toml) * canonical/charmlibs → fail ([tool.uv] present, no-build missing) * canonical/charm-ubuntu on chore/modernize-charm → pass * synthesised no-build = true → pass * synthesised no-build = false → fail * synthesised no-build = true + per-pkg override → pass with the override reflected in evidence

…ical shape Rewrites the template to match the canonical shape from OP0xx and the configs merged into canonical/operator, canonical/pebble, canonical/jubilant, etc: uv (not pip) as the default Python ecosystem, monthly schedule, labels, open-pull-requests-limit 100, commit-message prefix "chore", cooldown with semver-major-days 14, and the four canonical groups (charm-tech, dev-tooling, test-deps, runtime with minor+patch filter) plus an actions group for GHA. Replaces the tiny commented gomod stub with a full canonical gomod block. Updates add-dependabot.py's post-copy message to reflect the new customization steps. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

The sweep-history.md reference file was removed; strip the pointers to it from SKILL.md, README.md, and the check/fix script docstrings. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Rework the trusted-publishing template to match fleet convention (publish-pypi env, SHA-pinned actions, zizmor-clean shape) and split by tier for SBOM handling: - personal/canonical: inline CycloneDX SBOM + dual attestation (attest-build-provenance + attest-sbom). - product: add trusted-publishing-product.yml.template that delegates to a reusable sbom-secscan.yaml workflow, with sbomber manifests for sdist and wheel artefacts. Update SKILL.md and the trusted-publishing check's human-review guidance to route by tier and require the agent to modernise SHA pins to current release SHAs before committing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Co-authored-by: Tony Meyer <tony.meyer@gmail.com>

Every other workflow/template in this skill and in the actual repos (canonical/operator etc.) uses .yaml; the two trusted-publishing templates were the only .yml stragglers. Renames the files and updates the references in SKILL.md, the check script, and the intra-template pointers. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

The first cross-repo rollout of the uv hardening pattern (2026-07-03) surfaced that a global `[tool.uv].no-build = true` also refuses to build the workspace project's own editable install during `uv sync` / `uv run`, and uv has no allow-list or workspace-exempt flag. The deny-list workaround `no-build-package = [<every dep>]` is unmaintainable and silently regresses the moment a new sdist-only dep lands (the very case no-build was defending against). The fleet scan (2026-07-02) showed 0/571 deps are sdist-only, so no-build's current benefit is zero — it was purely defensive. The remaining pattern (`--locked` + `exclude-newer = \"7 days\"`) already gives a week's warning before any new dep enters resolution. Record the decision in references/decisions.md, drop the uv-no-build check, and remove its row from the SKILL.md check table. Revisit when uv gains an allow-list, workspace-exempt flag, or per-source override.

The rollout paired `exclude-newer = "7 days"` (rolling supply-chain quarantine) with `uv run --locked` (lockfile freshness enforcement). CI failed daily with the rolling `exclude-newer` value diverging from the absolute timestamp uv records in `uv.lock`, invalidating `--locked`. The two are fundamentally incompatible. Rolling `exclude-newer` is the security control (coverage on manual `uv add` / CI re-resolves that Dependabot cooldown misses); `--locked` is a hygiene gate that can be reconstructed by a PR-time structural check later. Drop `--locked`. Record the decision in references/decisions.md, drop the uv-locked-in-ci check, and remove its row from the SKILL.md check table. Revisit with a PR-time `pyproject.toml`↔`uv.lock` structural check when appetite exists.

`actions/attest-sbom` is deprecated in favour of `actions/attest`, which takes the same `subject-path` / `sbom-path` inputs and produces the same SBOM predicate. attest-sbom now runs as a thin wrapper over attest and will be sunset. Add a check that flags any workflow still on the old action so agents can propose the mechanical swap. Sweep across the charm-tech team turned up only canonical/hyrum; PR switching that repo will be linked from sweep-history once open. Ref: https://github.com/actions/attest-sbom (deprecation notice).

The no-build decision was recorded before the --locked one, so its text still referred to "the remaining pattern is --locked + exclude-newer". After the --locked decision, only exclude-newer survives — align the wording.

Replace the bare `[tool.uv]` block with the canonical fleet comment (rolling quarantine rationale) and drop `no-build = true`. See canonical/pytest-jubilant#98 for the exemplar PR and canonical/charm-tech#22 (references/decisions.md) for the recorded deferral of `no-build` and `--locked`: uv has no allow-list to exempt the workspace project from `no-build`, and rolling `exclude-newer` is fundamentally incompatible with `--locked`. Rolling `exclude-newer` is the surviving pattern.

Replace the `[tool.uv]` block with the canonical fleet comment (rolling quarantine rationale), drop `no-build = true`, and drop `--locked` from CI workflow `uv run` / `uv sync` invocations. See canonical/pytest-jubilant#98 for the exemplar PR and canonical/charm-tech#22 (references/decisions.md) for the recorded deferral of `no-build` and `--locked`: uv has no allow-list to exempt the workspace project from `no-build`, and rolling `exclude-newer` is fundamentally incompatible with `--locked` (uv records the resolved-at-lock-time timestamp in `uv.lock`, so every day CI recomputes a different value and `--locked` errors). Rolling `exclude-newer` is the surviving pattern.

…b-actions template The GitHub Actions ecosystem supports only cooldown.default-days; semver-major-days / semver-minor-days / semver-patch-days are supported on the uv/pip/gomod ecosystems but not github-actions. Leaving the key in fails the .github/dependabot.yaml validation check on every Dependabot PR, which in turn causes Dependabot to fall back to per-package PRs (no groups). Ref: https://docs.github.com/en/code-security/reference/supply-chain-security/dependabot-options-reference

tonyandrewmeyer changed the title ~~feat(charm-tech-baseline): add repo-setup audit skill~~ feat: add repo-setup audit skill Jul 2, 2026

Merge branch 'main' into feat/add-charm-tech-baseline-skill

4817a67