Skip to content

Publishing rehearsal: rename to agentguard, TestPyPI dry run, out-of-band settings checklist #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
amic25 merged 3 commits into from
Jul 31, 2026
Merged

Publishing rehearsal: rename to agentguard, TestPyPI dry run, out-of-band settings checklist #20

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a42fff1
Rename the distribution to agentguard while it is still free
claude Jul 30, 2026
feba245
Add a TestPyPI rehearsal, and stop a dispatch reaching production
claude Jul 30, 2026
6f1f914
Write down the settings CI cannot see, and why containment is per-con…
claude Jul 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 55 additions & 5 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@ name: Release
on:
push:
tags: ["v*"]
# Dispatch publishes to TestPyPI only. There is deliberately no input selecting the
# target: a dry run that can be pointed at production PyPI by choosing the wrong
# dropdown entry is a worse hazard than the one it exists to remove.
workflow_dispatch:

permissions:
contents: read
Expand All @@ -24,14 +28,28 @@ jobs:
# The tag is the only thing a human types in this pipeline, and the version is
# written in exactly one place. If they disagree, PyPI would receive a package
# declaring a version nobody tagged, permanently.
# Runs on both triggers: the declared and installed versions must agree whether or
# not a tag is involved, and a dry run is the cheapest place to catch a disagreement.
- name: Declared and installed versions must agree
run: |
declared="$(python -c 'import tomllib,pathlib; print(tomllib.loads(pathlib.Path("pyproject.toml").read_text())["project"]["version"])')"
installed="$(python -c 'import importlib.metadata as m; print(m.version("agentguard"))')"
echo "pyproject=$declared installed=$installed"
if [ "$declared" != "$installed" ]; then
echo "::error::pyproject declares '$declared' but the installed distribution is '$installed'"
exit 1
fi

# Tag-only: on a workflow_dispatch, GITHUB_REF_NAME is a branch name, and comparing a
# branch to a version would fail every dry run for the wrong reason.
- name: Tag must match the declared version
if: startsWith(github.ref, 'refs/tags/')
run: |
tag="${GITHUB_REF_NAME#v}"
declared="$(python -c 'import tomllib,pathlib; print(tomllib.loads(pathlib.Path("pyproject.toml").read_text())["project"]["version"])')"
installed="$(python -c 'import importlib.metadata as m; print(m.version("agentguard-sast"))')"
echo "tag=$tag pyproject=$declared installed=$installed"
if [ "$tag" != "$declared" ] || [ "$tag" != "$installed" ]; then
echo "::error::tag '$tag' disagrees with pyproject ('$declared') / installed ('$installed')"
echo "tag=$tag pyproject=$declared"
if [ "$tag" != "$declared" ]; then
echo "::error::tag '$tag' disagrees with pyproject ('$declared')"
exit 1
fi

Expand Down Expand Up @@ -61,12 +79,43 @@ jobs:
name: distributions
path: dist/

# Rehearsal. Exercises what a local install cannot: PyPI's metadata validation on
# receipt, the OIDC token exchange, and how the project page renders. It validates the
# *mechanism*; TestPyPI has its own trusted-publisher configuration, so a green run here
# says nothing about whether the production publisher is configured correctly.
publish-testpypi:
needs: build
if: github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
environment:
name: testpypi
url: https://test.pypi.org/p/agentguard
permissions:
id-token: write
steps:
- uses: actions/download-artifact@v8
with:
name: distributions
path: dist/
- uses: pypa/gh-action-pypi-publish@release/v1
with:
repository-url: https://test.pypi.org/legacy/
# A version can only be uploaded once. Without this, the second rehearsal of any
# given version fails on a conflict that says nothing about the release. Set here
# and deliberately not on the production job, where a conflict is a real signal.
skip-existing: true

publish:
needs: build
# Both halves are load-bearing. A workflow_dispatch can be run against a *tag* ref, so
# a tag-only condition would let a manual dispatch reach production PyPI - which is the
# one action in this pipeline that cannot be undone. Requiring the push event as well
# means production is reachable only by pushing a tag.
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/agentguard-sast
url: https://pypi.org/p/agentguard
permissions:
id-token: write
steps:
Expand All @@ -78,6 +127,7 @@ jobs:

github-release:
needs: publish
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-latest
permissions:
contents: write
Expand Down
36 changes: 18 additions & 18 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,20 @@ All notable changes follow [Keep a Changelog](https://keepachangelog.com/en/1.1.

### Changed

- **The distribution is named `agentguard`, not `agentguard-sast`.** Renamed before first
publication, so no package under the old name has ever existed on PyPI — early git history
referencing `agentguard-sast` describes a name that was never published, not one that was
retired. The import package and the CLI entry point were already `agentguard` and are
unchanged; only the name you `pip install` is affected, and only for anyone who built from
source before this release.
- **Exit code semantics.** A scan that does not complete now exits `2` where some cases previously
exited `0`. Automation that treated `0` as "clean" was previously being misled; it is now correct.
Files skipped by declared policy (`max_file_size_kb`) remain non-failing.
- **Config schema.** `plugins`, `disabled_rules`, and `severity_overrides` are no longer accepted in
a `.agentguard.yml` discovered in the repository under scan; they now require an explicit
`--config`. Migration: pass `--config .agentguard.yml` to keep the previous behaviour for a
repository you own, or move those keys to an operator-supplied file. `agentguard init` now emits
the repository-safe subset.
- **Rules declare their context; the engine enforces it.** `RuleMetadata` gained
`languages` (required), `ignore_regions`, `require_nodes`, and `fixture_policy`. Language
gating, comment/docstring/annotation awareness, node-kind gating, and test-fixture
Expand All @@ -35,6 +49,10 @@ All notable changes follow [Keep a Changelog](https://keepachangelog.com/en/1.1.

### Fixed

- A rule that raised on every file, or a file that could not be decoded, previously produced exit
code `0` — indistinguishable from a clean scan, so CI reported green with zero coverage. The exit
code now honours the same invariant that SARIF `executionSuccessful` already reported. Exit `2`
outranks the `--fail-on` threshold.
- `AG002` read any `.exec()` or `.eval()` method as the builtin, because call-name
resolution returned the bare attribute when the receiver was not a plain name.
`super().exec(*command)` was reported as critical arbitrary code execution.
Expand Down Expand Up @@ -83,24 +101,6 @@ All notable changes follow [Keep a Changelog](https://keepachangelog.com/en/1.1.
can only tighten a scan. The same fix closes repository control over `follow_symlinks`,
`max_file_size_kb`, and `exclude`.

### Fixed

- A rule that raised on every file, or a file that could not be decoded, previously produced exit
code `0` — indistinguishable from a clean scan, so CI reported green with zero coverage. The exit
code now honours the same invariant that SARIF `executionSuccessful` already reported. Exit `2`
outranks the `--fail-on` threshold.

### Changed

- **Exit code semantics.** A scan that does not complete now exits `2` where some cases previously
exited `0`. Automation that treated `0` as "clean" was previously being misled; it is now correct.
Files skipped by declared policy (`max_file_size_kb`) remain non-failing.
- **Config schema.** `plugins`, `disabled_rules`, and `severity_overrides` are no longer accepted in
a `.agentguard.yml` discovered in the repository under scan; they now require an explicit
`--config`. Migration: pass `--config .agentguard.yml` to keep the previous behaviour for a
repository you own, or move those keys to an operator-supplied file. `agentguard init` now emits
the repository-safe subset.

## [0.1.0] - 2026-07-16 — NEVER PUBLISHED

> **This release does not exist.** The entry below was written in advance and the release
Expand Down
6 changes: 3 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Offline. Never executes the code it scans. Designed to be run on repositories yo
not read.

```bash
pipx install agentguard-sast
pipx install agentguard
agentguard scan ./project
```

Expand Down Expand Up @@ -122,10 +122,10 @@ AgentGuard requires Python 3.10 or newer.

```bash
# Isolated CLI installation (recommended)
pipx install agentguard-sast
pipx install agentguard

# Or with pip
python -m pip install agentguard-sast
python -m pip install agentguard

# From source
git clone https://github.com/amic25/agentguard.git
Expand Down
212 changes: 212 additions & 0 deletions WORKLOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1604,3 +1604,215 @@ Either way this is a decision about a namespace you own, so it stays with you.
- The `[0.1.0] — NEVER PUBLISHED` annotation untouched.
- PRs #12, #13, #14, #15 untouched. #14 adds a pattern to the unbounded AG001 and will be
gated by the linearity job on its own PR now that `--check` runs in CI.

---

## Unit 21 — distribution renamed to `agentguard` — 2026-07-30

Status: complete
Changed: `pyproject.toml`, `src/agentguard/__init__.py`, `tests/test_cli.py`,
`.github/workflows/release.yml`, `README.md`, `docs/launch/REDDIT.md`, `CHANGELOG.md`

### Nine sites, not six

The brief listed six. Post-#19 there are **nine**, and the three additions are the ones
that would have failed loudest:

```
pyproject.toml:6 name
src/agentguard/__init__.py:16 importlib.metadata lookup <- added by #19
tests/test_cli.py:65 metadata assertion <- added by #19
.github/workflows/release.yml:31 tag/version check <- added by #19
.github/workflows/release.yml:69 PyPI environment URL
README.md:12, 125, 128 install instructions
docs/launch/REDDIT.md:10 install instruction
```

Renaming without the three from #19 would have left `__version__` falling back to
`0.0.0+unknown`, so `--version`, the JSON `tool.version`, and the SARIF driver version
would all have reported a version that does not exist — and the release `verify` job's tag
check would have failed on a `PackageNotFoundError` after the tag was pushed. Precisely the
class of failure #19 was written to prevent, reintroduced by a rename that looked textual.

Seven references remain in `WORKLOG.md` and are deliberate: that file is the historical
record, and the old name is part of it.

### Verified after a clean reinstall

A stale editable install would have masked a broken metadata lookup, so the old
distribution was uninstalled first:

```
distribution -> agentguard
__version__ -> 0.2.0
agentguard --version -> AgentGuard 0.2.0
version("agentguard-sast") -> PackageNotFoundError (correct: the old name is gone)
built artifacts -> agentguard-0.2.0.tar.gz, agentguard-0.2.0-py3-none-any.whl
twine check -> PASSED (both)
wheel metadata -> Name: agentguard, License-Expression: Apache-2.0
wheel package dir -> agentguard/ (import name unchanged, as expected)
clean-venv install -> pip show name: agentguard, --version: AgentGuard 0.2.0
pytest -> 198 passed, 1 xfailed
bench -> 33 of 34 behave as labelled
linearity gate -> exit 0
```

### CHANGELOG had two `### Changed` and two `### Fixed` under `[Unreleased]`

Not caused by this unit — accumulated across earlier commits, each appending its own
section rather than merging into the existing one. Adding the rename note made it three,
which is how it surfaced. Consolidated to one of each in Keep a Changelog order (Added,
Changed, Fixed, Removed, Security); all four affected bullets verified still present, none
dropped in the move.

Bench delta: none — a distribution name change touches no rule.
Decisions taken alone:
1. **`WORKLOG.md` left untouched.** It records what was true at the time, and rewriting it
to say `agentguard` would make the earlier entries wrong.
2. **CHANGELOG sections consolidated** rather than left duplicated, since the file is
about to be read by anyone evaluating a first release.
Next: unit 2, the TestPyPI dry run.

---

## Unit 22 — TestPyPI dry run — 2026-07-30

Status: complete (job added; **the run itself needs a publisher I cannot configure** — see
the residual gap below)
Changed: `.github/workflows/release.yml`

`workflow_dispatch` now publishes to TestPyPI through the same trusted-publishing shape as
production: `id-token: write`, a named environment, `pypa/gh-action-pypi-publish`, and the
same `verify → build` gate. Only the `repository-url` and the environment differ.

```
job needs runs when
verify - both triggers
build verify both triggers
publish-testpypi build workflow_dispatch only
publish build push AND refs/tags/**
github-release publish push AND refs/tags/**
```

### A hole found while writing it

`workflow_dispatch` can be run against a **tag** ref, not only a branch. With production
gated on `startsWith(github.ref, 'refs/tags/')` alone, a manual dispatch on a tag would
have satisfied it and published to production PyPI — the one action in this pipeline that
cannot be undone, reachable from a dropdown. Production now also requires
`github.event_name == 'push'`. Evaluated across every combination:

```
event ref TestPyPI production
push refs/tags/v0.2.0 False True
workflow_dispatch refs/heads/main True False
workflow_dispatch refs/tags/v0.2.0 True False <- the hole, now closed
```

There is deliberately **no input**选 selecting the publish target. A dry run that can be
aimed at production by picking the wrong dropdown entry is a worse hazard than the one it
removes.

### Two adjustments the dry run forced

- **The tag check had to be split.** On a dispatch, `GITHUB_REF_NAME` is a branch name, so
comparing it to a version would fail every dry run for the wrong reason. It is now two
steps: *declared vs installed* runs on both triggers (a disagreement is worth catching in
a rehearsal), and *tag vs declared* is guarded by `startsWith(github.ref, 'refs/tags/')`.
- **`skip-existing: true` on the TestPyPI job only.** TestPyPI accepts a version once, so
the second rehearsal of `0.2.0` would otherwise fail on a conflict that says nothing
about the release. Deliberately not set on production, where a version conflict is a real
signal that something is wrong.

### Residual gap — stated plainly

**A green dry run validates the mechanism, not the production configuration.** TestPyPI
holds its own trusted-publisher record, separate from PyPI's. A successful TestPyPI publish
proves the workflow shape, the OIDC exchange, metadata acceptance, and page rendering — and
proves *nothing* about whether the PyPI publisher for `agentguard` names the right owner,
repository, workflow filename, and environment. That one is verifiable only by inspection,
which is unit 23.

**I have not run the dispatch.** It needs a TestPyPI project with a trusted publisher for
this repository and a GitHub environment named `testpypi`, both of which are yours. Once
they exist: Actions → Release → Run workflow, from any branch.

After it runs, the page checks worth doing are: README rendering as Markdown rather than
raw markup, `License-Expression: Apache-2.0` showing as a licence badge rather than the
wall of text the pre-#19 metadata would have produced, and the four project URLs resolving.

Bench delta: none — workflow only.
Decisions taken alone:
1. **No target input on the dispatch.** Rejected a `choice` input for production-vs-test:
the failure mode it introduces is worse than the flexibility it buys.
2. **`skip-existing` on TestPyPI only**, reasoning above.
Next: unit 23, documenting the settings CI cannot see.

---

## Unit 23 — settings outside version control, and corpus containment as a decision — 2026-07-30

Status: complete
Changed: `docs/GITHUB_SETUP.md`, `docs/DECISIONS.md`

### Three live problems found while writing the checklist

Checking each setting rather than describing it turned up three things wrong right now:

1. **No GitHub environments exist.** `gh api .../environments` returns `[]`, while
`release.yml` references `pypi` and now `testpypi`. GitHub creates one implicitly on
first use, so a publish would still succeed — but with **no protection rules**, meaning
anyone able to push a tag can publish. And if the PyPI publisher record names an
environment, the names must match exactly or the OIDC exchange fails.
2. **The About field still advertises "vulnerable dependencies".** AG009 was deleted in
#17; the repository page has claimed the capability ever since. Not fixed here — it is
a setting, and settings are yours — but it is the clearest possible demonstration of why
the checklist exists: nothing in CI renders that string, so nothing could fail.
3. **`GITHUB_SETUP.md` was recommending the mistake.** Line 6 told the reader to require
the `test` and `package` checks. `test` is one of the three phantom contexts that
blocked every merge for two sessions. The document that onboards a maintainer was
teaching them to reproduce the incident. Corrected, with the reason attached.

### What the checklist covers

Six surfaces, each with what depends on it, a verify command, and whether the breakage is
recoverable: PyPI trusted publisher (**unrecoverable** — fails after the tag is spent),
the `pypi`/`testpypi` environments, branch-ruleset required contexts, About and topics,
dependency graph and Dependabot alerts, and the social preview upload.

Live state recorded at the time of writing:
```
environments [] <- neither exists
required contexts ["CodeQL","build","ci-ok"] <- all real check-run names
vulnerability-alerts 204 No Content <- enabled
About "...vulnerable dependencies..." <- stale, AG009 is gone
```

Plus a five-item pre-tag sequence, ordered so the one unrecoverable item is checked last
and checked twice.

### Two `DECISIONS.md` entries

**"A CI-verifiable repository still has a configuration surface CI cannot see."** The
honest framing is that an inspection checklist is a weak control — manual, staleable, only
run by someone who remembers. It is there because the alternative is nothing. Four
incidents, and the shape is identical every time: *the repository is green, and the thing
that is wrong is not in the repository*. The entry also records the one real mitigation —
pulling settings into version control where possible, which is what `ci-ok` does by
converting a branch-protection problem into a workflow problem.

**"Corpus containment is per-consumer, because no single boundary holds."** Four readers,
no two sharing a mechanism, and the directory irrelevant to all four — `tests/corpus/` is a
convention this project observes and nothing else does. The standing rule is stated: any
file whose shape implies a role will be interpreted by something regardless of location,
and new corpus files of those shapes get checked against the reader list before landing.
Both existing mitigations are noted as non-generalising.

Verified: 198 passed + 1 xfailed, lint, mypy, bench 33 of 34.
Bench delta: none — documentation only.
Decisions taken alone:
1. **Did not change the About field.** It is a setting and settings are yours; flagged
instead, with the corrected wording supplied in `GITHUB_SETUP.md` so it can be pasted.
2. **Corrected the stale branch-protection recommendation** rather than only documenting
the general rule — a document actively recommending a known failure is worse than one
that is silent.
Loading
Loading