fix(ci): stop exempting the gate's own fixture corpus from full-gate-build - #151
Merged
Merged
Conversation
added 2 commits
August 15, 2026 20:23
…build docs_only_exemption:true let the shared classifier's *.md pattern treat examples/*/content/**.md as documentation, so a diff confined to typikon's own fixture corpus (PR #120) skipped full-gate-build with three green checks. The classifier is a shared reusable and cannot be parameterized per-repo, so the local predicate is now false: full-gate-build always runs here. ci/check-fixture-corpus-exemption.sh proves the corpus-only diff both matches the docs-only pattern (control) and is no longer exempt under this repo's configured value (regression guard).
…eaders 'WHAT this proves, as a negative case:' and 'Usage:' were freeform prose outside the closed comment-tag set (STANDARDS.md Structured comment tags). Retagged both as NOTE: -- the closest fit among WHY/WARNING/NOTE/PERF/SAFETY/INVARIANT for non-design-rationale context.
This was referenced Aug 16, 2026
forkwright
pushed a commit
that referenced
this pull request
Aug 16, 2026
🤖 I have created a release *beep* *boop* --- ## [0.4.0](v0.3.0...v0.4.0) (2026-08-16) ### Features * **schemas:** add a fail-closed consumer schema registry ([#154](#154)) ([3d9432a](3d9432a)) * **templates:** make the favicon path configurable like every sibling brand asset ([#141](#141)) ([729ce2e](729ce2e)) ### Bug Fixes * **ci:** stop exempting the gate's own fixture corpus from full-gate-build ([#151](#151)) ([8690c82](8690c82)) * **templates:** give the home page a genuine top-level heading ([#152](#152)) ([880f711](880f711)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
forkwright
added a commit
that referenced
this pull request
Aug 17, 2026
…d claim-accuracy defects (#171) ## Summary Fixes 16 of the 18 findings in #92; the other 2 were already fixed on `main` by prior PRs (evidence below, per `#5a` in the fleet agent brief). One additional fix (JSON-LD script-injection escaping) generalizes beyond the single file the issue cited to all six `partials/ld-*.html` partials, since every sibling shared the identical unescaped `json_encode` pattern — fixing only the named file would have left the same defect live in five others. ## Acceptance criteria Issue #92: `Done when: every bullet is either fixed, or closed with a stated reason it is not a defect.` ### `bin/` (7) 1. **`bin/typikon-check:65` — `base_host()` single-quoted TOML.** Fixed — `bin/typikon-check:103-112`. `base_host()`'s `sed` now matches both `"..."` and `'...'` TOML string forms. Verified: piping a single-quoted `base_url = 'https://example.com'` through the pre-fix pipeline produced `base_url = 'https:` (garbage); through the fixed pipeline, `example.com` for both quote styles. 2. **`bin/typikon-check:7` — "fail-fast" header claim.** Already fixed — `bin/typikon-check:7-9` currently reads "Runs every gate stage in order. Stages do NOT stop at the first failure." Fixed in PR #103 (`3. The header claimed fail-fast; it isn't`). No change in this PR. 3. **`bin/typikon-init:107` — "always refresh" comment vs skip-if-exists code.** Fixed — `bin/typikon-init:131-137`. Corrected the comment: these files are create-once (matches `typikon-refresh`'s own documented contract, which explicitly does NOT re-render `_headers`/`_redirects`). 4. **`bin/typikon-init:91` — swallowed theme-submodule refresh failure.** Fixed — `bin/typikon-init:90-97`. A failed `git pull --ff-only` now prints `warn themes/typikon: refresh failed, continuing with existing checkout: <git's own error>` to stderr instead of `|| true`-swallowing it silently. Still non-fatal (an offline forge on a re-run must not abort the whole scaffold). 5. **`bin/typikon-validate:138` — JSON Pointer root emits `"/"` not `""`.** Fixed — `bin/typikon-validate:420-428`. A root-level failure (`err.absolute_path` empty) now emits `pointer: ""` per RFC 6901. 6. **`bin/typikon-validate:155` — `load_schemas()` raw traceback on malformed schema.** Fixed — `bin/typikon-validate:493-499` and `:592-599` (both the per-slug loop and the `.core.schema.json` loop carried the identical unwrapped `json.loads()`). Both now catch and report `{"error": "malformed schema <path>: <exc>"}` + exit 2. 7. **`bin/typikon-validate:29` — exit-code docstring contradicts implementation for malformed frontmatter.** Fixed — `bin/typikon-validate:57-64`. Malformed frontmatter is folded into the per-file `failures` list (exit 1), not the runtime-error class (exit 2); the docstring said the opposite. Corrected the doc; the code was already the intended behavior (same JSONL per-file reporting shape as a schema failure). ### `ci/` (4, + 1 bonus from the issue thread) 8. **`ci/csp-enforce.sh:46` — unbounded `grep`/argv, E2BIG risk.** Fixed — `ci/csp-enforce.sh:65-70`, `ci/csp-scan.py:106-124`. File paths now go over stdin (bash builtin `printf`, no `execve()` of its own) instead of argv; `csp-scan.py` reads newline-delimited paths from stdin when argv is empty. 9. **`ci/kanon-ci.toml.tmpl:40` — missing TLS pinning vs the GitHub template it claims to mirror.** Fixed — `ci/kanon-ci.toml.tmpl:41,75`. Both `curl` calls (zola, lychee) now carry `--proto '=https' --tlsv1.2`, matching `ci/github-workflow.yml.tmpl:64,110`. 10. **`ci/lychee.toml:12` — false cross-run cache-persistence claim.** Fixed — `ci/lychee.toml:11-16`. Corrected: neither CI template persists `.lychee_cache/` across runs today (verified: no `actions/cache` / forge equivalent in either template). `cache = true` stays on — it still dedupes repeat hits to the same URL within one run. 11. **`ci/playwright.config.ts:10` — header claims `public/`, gate actually serves `public-local/`.** Already fixed — `ci/playwright.config.ts:16-21` currently describes `public-local/` correctly, with an OS-allocated port (no hardcoded 8080 claim). Fixed in PR #116. No change in this PR. 12. *(Bonus, from the issue thread, not one of the 18 — fixed while here)* **`ci/check-consumer-check-extension.sh:32-33` — freeform `# Usage:` comment tag.** Fixed — `ci/check-consumer-check-extension.sh:32`, now `# NOTE: usage — ...`, per the fix already applied to the identical shape in PR #151. ### `schemas/` (1) 13. **`schemas/page.schema.json:66` — `og_image` extension allowlist inconsistent across schemas.** Fixed — `schemas/page.core.schema.json:64` and `schemas/journal-entry.schema.json:52` (the file split into `page.schema.json` + `page.core.schema.json` since the issue was filed; the actual `og_image` pattern lives in `page.core.schema.json` now). Both now accept `.jpeg`, matching `faq`/`sizing-guide`/`product` schemas, which already did. All 5 schemas' `og_image` patterns now agree. ### `templates/` (6) 14. **`templates/atom.xml:25` — `<updated>` from newest-DATED entry, not most-recently-EDITED entry.** Fixed — `templates/atom.xml:24-45`. Now scans every entry's own `updated`-or-`date` and takes the max (`set_global` + epoch-int comparison, since Tera's `>` only compares numbers). Verified against a real `zola build`: an older-dated entry given a newer `updated` now correctly wins. 15. **`templates/faq.html:36` — auto-slugify anchor collision.** Fixed — `templates/faq.html:42-59`. A colliding anchor (explicit or slugify-derived) gets a `-{{ loop.index }}` suffix instead of colliding on `id=`. Verified against a real `zola build`: two questions that previously rendered two `id="do-you-ship-internationally"` elements now render `id="do-you-ship-internationally"` + `id="do-you-ship-internationally-2"`. 16. **`templates/journal-entry.html:16` — hardcoded `og_type`.** Fixed — removed the override; `journal-entry.html` now inherits `page.html`'s `og_type` block (`templates/page.html:13`), which already reads `page.extra.og_type | default(value="article")` correctly. Verified: a real build's `og:type` meta still reads `article` by default with no regression. 17. **`templates/partials/ld-article.html:21` — hand-concatenated image/logo URLs instead of `get_url()`.** Fixed — `templates/partials/ld-article.html:20-28`, `templates/partials/ld-product.html:27-32`, `templates/partials/ld-organization.html:29-34` (the issue: "**three** JSON-LD partials"). All three now call `get_url(path=..., trailing_slash= false)`, matching the same field's resolution in `base.html`/`page.html`. Verified: with `base_url = "https://example.com/"` (trailing slash), hand-concatenation produced `https://example.com//img/og.png` (double-slash, broken); `get_url()` produces `https://example.com/img/og.png`. 18. **`templates/partials/ld-organization.html:14` — no `assert::required` guard, unlike every sibling.** Fixed — `templates/partials/ld-organization.html:13,26`. Now imports `partials/assert.html` and asserts `config.extra.logo_path`, matching every sibling `ld-*.html`. Verified: pre-fix, an absent `logo_path` still failed the build, but with a bare Tera engine error ("Variable `config.extra.logo_path` not found in context") naming neither the field's role nor which template needs it; post-fix, the same missing field now fails with `partials/ld-organization.html: missing required metadata` — the same clear, actionable message every sibling partial already gives. 19. *(the issue's `ld-organization.html:19` finding — same file, listed separately below since it's a distinct defect)* **JSON-LD `</script>` breakout via unescaped `json_encode`.** Fixed — new `templates/partials/ld-json.html` (shared `ldjson::str()` macro, escapes `/` → `\/`, RFC 8259 §7-legal) wired into **all six** `partials/ld-*.html` (article, breadcrumb, faq, howto, organization, product) — not only the file the issue cited, since every sibling shared the identical `json_encode | safe` pattern with no escaping. Verified: a field value containing `"Hello</script><script>alert(1)</script>"` round-trips through `json.loads()` to the exact original string, and the literal substring `</script` no longer occurs anywhere in the `<script type="application/ld+json">` payload. Note: fixing `ld-organization.html`'s `assert::required` call surfaced a real cross-file Tera scoping gap — `base.html` (which `{% include %}`s `ld-organization.html` inside a block every page reaches via `super()`) did not itself import `partials/assert.html`, and Tera resolves a macro namespace by which template an `include` tag is textually written in, not by the included partial's own imports, once reached via `extends` + `super()`. Fixed by importing `assert` in `templates/base.html:20` too; documented + verified (see commit message). **18/18 accounted for**: 16 fixed in this PR, 2 already fixed on `main` (items 2 and 11 above) with cited evidence. ## Negative fixtures > Negative fixture: `bin/typikon-check` (`base_host()`) — watched failing by > piping a single-quoted `base_url = 'https://example.com'` config line > through the pre-fix `sed` pipeline, which produced the literal garbage > `base_url = 'https:` before the fix and `example.com` after (both quote > styles), run against the exact shipped `sed` expression from each > revision. > Negative fixture: `bin/typikon-validate` (root JSON Pointer) — watched > failing by running `origin/main`'s own shipped `bin/typikon-validate` > against a fixture page missing the required `title` field, which produced > `"pointer": "/"` before the fix; the current worktree's shipped > `bin/typikon-validate` against the identical fixture produces > `"pointer": ""` after. > Negative fixture: `bin/typikon-validate` (`load_schemas()` crash) — > watched failing by running `origin/main`'s own shipped > `bin/typikon-validate` against a schema directory with `faq.schema.json` > truncated mid-object (invalid JSON), which produced a raw > `json.decoder.JSONDecodeError` Python traceback (exit 1) before the fix; > the current worktree's shipped `bin/typikon-validate` against the > identical corrupted schema produces a clean > `{"error": "malformed schema ...: Unterminated string ..."}` (exit 2) > after. > Negative fixture: `bin/typikon-init` (swallowed refresh failure) — > watched failing by extracting the exact shipped refresh line from > `origin/main`'s `bin/typikon-init` (`(cd themes/typikon && git pull > --ff-only origin main 2>/dev/null) || true`) and running it verbatim > against a `themes/typikon` checkout whose `origin` remote is > unreachable, which produced silent success (exit 0, empty stdout AND > stderr) before the fix; the current worktree's shipped replacement lines > against the identical unreachable remote produce exit 0 (still > non-fatal, correctly) but with `warn themes/typikon: refresh failed, > continuing with existing checkout: fatal: ... does not appear to be a > git repository ...` on stderr after. > Negative fixture: `templates/atom.xml` (`<updated>`) — watched failing > by building `examples/sample-blog` (real `zola build`, shipped > templates) with `origin/main`'s `templates/atom.xml` against a fixture > where the older-dated journal entry has a newer `updated` than the > newer-dated entry's own value, which produced `<updated>2026-02-10T00:00:00+00:00</updated>` > (the newer entry's own date, ignoring the true most-recent edit) before > the fix; the current worktree's `templates/atom.xml` against the > identical fixture produces `<updated>2026-08-10T00:00:00+00:00</updated>` > (the actual most-recent edit) after. > Negative fixture: `templates/faq.html` (anchor collision) — watched > failing by building a FAQ page (real `zola build`, shipped templates) > with two questions that slugify identically, against `origin/main`'s > `templates/faq.html`, which produced two `id="do-you-ship-internationally"` > elements before the fix; the current worktree's `templates/faq.html` > against the identical fixture produces `id="do-you-ship-internationally"` > + `id="do-you-ship-internationally-2"` after. > Negative fixture: `templates/partials/ld-organization.html` > (`assert::required` + `get_url()`) — watched failing by building > `examples/sample-blog` (real `zola build`) with `config.extra.logo_path` > removed from `config.toml`, against `origin/main`'s > `templates/partials/ld-organization.html`, which produced a bare Tera > engine error (`Variable 'config.extra.logo_path' not found in context`) > before the fix; the current worktree's version against the identical > fixture produces `partials/ld-organization.html: missing required > metadata 'config.extra.logo_path'` after — matching the message shape > every sibling `ld-*.html` partial already uses. > Negative fixture: every `partials/ld-*.html` (JSON-LD escaping) — > watched failing by piping `"Hello</script><script>alert(1)</script>"` > through the pre-fix `value | json_encode | safe` pattern (shared, > unescaped, across all six partials), which left the literal substring > `</script` intact in the `<script type="application/ld+json">` payload; > the current worktree's `partials/ld-json.html::str()` macro against the > identical input produces `Hello<\/script><script>alert(1)<\/script>` — > `json.loads()`-equivalent to the original string, with `</script` > nowhere in the byte stream. All 16 real JSON-LD blocks rendered across > both `examples/*` sites (blog + shop, every template) were re-verified > to still `json.loads()` cleanly after the change. All ten fixtures ran the actual shipped file content at each git revision (`origin/main` vs this branch) — extracted verbatim (`git show origin/main:<path>`) or symlinked directly at the theme root for a real `zola build` — never a re-implementation. ## Verification - `kanon lint <path> --all` — zero violations across all 21 changed/added files (individually checked). - `python3 bin/typikon-validate examples/sample-blog` / `examples/sample-shop` — both `{"failed": 0}` against the updated schemas. - Real `zola build` + `zola check` — both example sites build and check clean. - `ci/check-xml-output.sh`, `ci/csp-enforce.sh` — both example builds pass. - Every `ci/check-*.py` / `ci/check-*.sh` regression script in the repo (triad-schema, consumer-schema-registry, page-template-cascade, font-coverage, release-config, asset-provenance, favicon-path, init-favicon-path, asset-path-existence, init-staging-scope, control-contrast, home-heading, content-heading-collision, interactive-contrast (+selftest), workflow-template, consumer-check-extension, fixture-corpus-exemption, deploy-bundle-gate, cf-deploy-gate) — all pass. - `TYPIKON_CHECK_MODE=dev bin/typikon-check examples/sample-blog` and `.../sample-shop` — every stage passes except `pa11y` (`pa11y-ci` genuinely absent on this box; `lychee` and `playwright-smoke`, both present, pass real link-checking and browser smoke tests against the fixed templates). Not run: GitHub Actions CI (this box does not run cargo/full builds per fleet policy; this repo's own CI is zola/python/node-based and this PR's branch push will trigger it — this repo is public, so Actions minutes are unmetered). Closes #92 Co-authored-by: forkwright <cody@forkwright.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Finding
.github/workflows/gate-attestation.yml:137passeddocs_only_exemption: truetoforkwright/.github'shybrid-gate.ymlreusable, pinned at84a39d3344660e4010c32755b33a6a88912ad929. That reusable'scheck-trailerjob, inits "Check for docs-only changeset" step, classifies any diff where every changed
path matches
*.md/docs/*/llms.txtas docs-only and skipsfull-gate-buildentirely. The step runs
hybrid-gate.yml:155-219at the pinned SHA (:205-276atcurrent
forkwright/.githubmain); the case-pattern match itself is at:208pinned/
:266main — verified both bygit show <ref>:.github/workflows/hybrid-gate.ymlagainst the pinned SHA and
origin/main. The pattern-match arm (*.md|docs/*|llms.txt)is byte-identical between the two; only the diff-range computation feeding it changed
(two-dot
origin/${BASE_REF}..HEADat the pinned SHA → three-dot...HEADon main,an unrelated merge-base-vs-moving-base fix), which does not change what counts as a
match for a fixed diff like this repo's.
examples/*/content/**.mdmatches*.md,but it is not documentation: it is the fixture corpus
ci/run-fixtures.shbuilds andasserts against via
bin/typikon-check(ci/run-fixtures.sh:16-17). A diff confinedto that corpus is therefore classified exempt, when it is exactly the diff most in
need of the gate.
Evidence
PR #120 touched only two fixture-corpus files:
Run 31339059510:
ai-attributionsuccess,check-trailersuccess,gatesuccess,full-gate-buildskipped. Three green checks and an aggregate
gatepass with the one job thatbuilds anything never having run.
Why this matters
A fixture is how this repo proves a claim. A build failure #120's fixture would have
introduced — a schema violation, a broken link, a pa11y regression — would have
reached
mainwith four green checks behind it, discovered only by the next PR thathappens to touch a
.sh/.tsfile. The classifier lives in a separate shared repo(
forkwright/.github) and keys on file extension alone; it has no notion of "gateinput" vs. "documentation" to parameterize per caller, so no path-level fix is
possible from this repo. Teaching the shared classifier to exclude gate-input globs
fleet-wide is out of scope here by design — issue #122 itself flags that as needing
deliberate review across every consumer, not a drive-by edit riding this fix.
Desired correction
.github/workflows/gate-attestation.yml:137now setsdocs_only_exemption: falsewith an inline comment recording why: no diff can be exempted here without the
shared classifier being able to distinguish
examples/**from real docs, which itcurrently cannot.
full-gate-buildnow runs on every PR to this repo regardless ofwhich paths changed — the repo is public, so this trades no metered CI budget.
ci/check-fixture-corpus-exemption.shis the negative-case fixture, wired intoci/run-fixtures.sh:15(so it runs as part of the gate's owncheck_cmd, not justlocally). It reads this repo's actual configured
docs_only_exemptionvalue out ofgate-attestation.yml(never hardcodes it) and, against PR #120's exact two-filediff:
(
*.md|docs/*|llms.txt), i.e. it is a faithful reproduction of docs-only exemption skips the gate for changes to the gate's own fixture corpus #122's known-badshape, not a pattern invented for this test.
hybrid-gate.ymlcomputes (
pattern_match && docs_only_exemption == "true") and asserts it isfalse.Ran directly against the un-fixed workflow (
docs_only_exemption: true), it fails:Against the fixed workflow in this PR, it passes:
A reviewer can disprove this by: reverting
docs_only_exemptiontotrueandshowing
ci/check-fixture-corpus-exemption.shstill passes (it will not — that isthe fixture's whole job), or by showing the two cited fixture paths no longer match
the shared classifier's pattern (they do — both end in
.md), or by showingci/run-fixtures.shdoes not actually invoke the new script (it does, at line 15).cargo fmtis not applicable (no Rust in this repo —fmt_cmd/clippy_cmd/nextest_cmdare all no-ops pergate-attestation.yml's own header comment).shellcheck ci/check-fixture-corpus-exemption.shandshellcheck ci/run-fixtures.share both clean.
kanon lint . --allreports zero findings against every file this PRtouches (one pre-existing, unrelated
PY/bare-exceptfinding inci/asset-provenance-scan.pypredates this branch and is untouched by it).On #122's literal Done-when
#122's Done-when reads: "a PR touching only
examples/*/content/**.mdrunsfull-gate-build, demonstrated by a run link." The evidence above is a localshell-script simulation of the classifier plus PR #120's old run (which shows the
bug, not the fix) — neither is that run link. This is not an oversight; it is a
sequencing constraint worth stating plainly rather than arguing around:
gate-attestation.yml's trigger ispull_request: branches: [main](:65-69) —GitHub filters
pull_requeston the PR's base branch, so the workflow only firesat all when a PR targets
main. For apull_requestrun, GitHub executes theworkflow file as it exists in the merge of head into base — which is why this PR's
own head already ran
full-gate-buildunder its owndocs_only_exemption: false(run 31921071233, this
PR's current head 059fc7d — all 4 jobs pass) even though
maindoesn't have thatvalue yet. But the PR diff GitHub
shows is always head vs. base: as long as
mainlacks this fix, any PR built on topof it that also touches only fixture content would carry this fix's own workflow and
script changes in that diff too — it would not be "a PR touching only
examples/*/content/**.md." There is no diff shape that is simultaneously (a)confined to fixture content and (b) evaluated against the fixed classifier unless the
fix is already merged to
main. The empirical demonstration #122 asks for istherefore only producible after this PR merges, not before it.
Plan, so this doesn't get silently dropped: merge this PR, then open a follow-up PR
against the now-fixed
maintouching only a fixture-corpus file (e.g. a no-opcontent edit to
examples/sample-shop/content/sizing.md), and post that run's linkas a comment on #122 before closing it. #122 is intentionally NOT auto-closed by this
PR (see below) so that comment is the actual closing action, not this merge.
Addresses #122 — left open; closed separately once the run-link comment above lands,
per the plan in "On #122's literal Done-when."