Skip to content

fix(gates)!: tee no longer masks exit codes; skip deps scan when unchanged - #11

Merged
craigmbooth merged 2 commits into
mainfrom
fix/gates-enforce-exit-codes-and-skip-unchanged-deps
Aug 1, 2026
Merged

fix(gates)!: tee no longer masks exit codes; skip deps scan when unchanged#11
craigmbooth merged 2 commits into
mainfrom
fix/gates-enforce-exit-codes-and-skip-unchanged-deps

Conversation

@craigmbooth

Copy link
Copy Markdown
Collaborator

The bug that makes this MAJOR

The gate suite could not fail. Every gate step is gate-cmd 2>&1 | tee /tmp/x.log. GitHub's implicit shell for a run: step is bash -e {0}no pipefail — so the pipeline's exit status is tee's, which is always 0.

Caught in the wild, intendent run 30638282003:

Found 10 known vulnerabilities, ignored 10 in 1 package
onnx 1.19.0  PYSEC-2026-104   1.21.0
... 9 more, none in that repo's allowlist

pip-audit exits 1 there (pkg_count != fixed_pkg_count → sys.exit(1)). The job reported success. The step log confirms the shell:

##[group]Run ".colormath-lib/scripts/audit-deps.sh" 2>&1 | tee /tmp/audit.log
shell: /usr/bin/bash -e {0}          <-- no pipefail

Zero gate jobs have failed across intendent's last 20 runs. gates.yml's own header asserted the opposite ("bash runs with -o pipefail, so tee never masks a real failure") — that assumption was never true; only an explicit shell: bash gets -eo pipefail.

Fix: workflow-level defaults.run.shell: bash. One declaration covers all 22 tee'd steps and any future one. Jobs set only defaults.run.working-directory, which merges with this per-key rather than replacing it.

The speed problem

The deps gate takes 5–7 min in intendent while every other gate finishes under 2m30s. Step timings say it isn't setup:

Step Duration
checkout + Python + install poetry/pip-audit 14s
Audit locked dependencies 6m16s

Inside that step, poetry export + startup is 1.7s; the rest is ~200 sequential per-package PyPI lookups. Measured locally on the same lockfile: 18s cold, 16s warm — so it's per-request latency from the Azure runners, not the tool. Caching won't help, and --vulnerability-service osv is both slower locally (42s) and returns a different finding set (21 vs 13), so it's not a drop-in.

The verdict is a pure function of three inputs: locked versions, allowlist, upstream advisory databases. A PR that moves neither of the first two can only differ from its base by the third — and the base branch re-scans in full on every merge. Only 4 of intendent's last 40 commits touched poetry.lock/pyproject.toml.

So: on pull_request, skip when the PR changes none of poetry.lock, pyproject.toml, .colormath/audit.conf. Every other event scans in full. New deps-skip-unchanged input (default true) opts out.

Verification

Skip logic tested against intendent's real history, simulating each PR shape:

Case Result
PR changes poetry.lock scan=true
PR changes only .colormath/audit.conf scan=true
PR changes only pyproject.toml scan=true
PR changes no dependency inputs scan=false
Base advanced with a lock bump, PR didn't touch deps scan=false (two-dot would have said true — hence ...)
push / schedule / workflow_dispatch scan=true
deps-skip-unchanged: false scan=true

Every uncertain path — base fetch fails, git errors — resolves to scan=true. A gate that cannot prove the deps are unchanged scans.

actionlint + shellcheck clean. example/'s lockfile audits clean today, so the deps gate stays green in the self-test.

Note on this PR's own CI: it changes no example/ dependency inputs, so the deps gate will skip here — that's the skip path demonstrating itself. I'll push a follow-up commit touching example/.colormath/audit.conf to exercise the scan path, then revert it, so both paths are proven before this is tagged.

The more interesting question is what pipefail newly surfaces in example/ across the other 21 gates. Per LIFECYCLE, anything red gets fixed in this PR before it can be tagged.

Release

  • colormath-ref default stamped v2.0.0v3.0.0
  • CHANGELOG entry with Upgrade notes (paste-into-Claude-Code block), including the known intendent fallout: 10 onnx advisories and 3 nltk ones (CVE-2026-12075/-12061/-12074, fixed in 3.10.0)
  • README documents the skip and the schedule: escape hatch

Tagging, and the talas → intendent → runwayz canary order, are yours to run — I have not tagged or merged anything.

craigmbooth and others added 2 commits August 1, 2026 17:15
…anged

Two changes to the deps gate's job, one of which turns out to apply to the
whole suite.

**The gates could not fail.** Every gate step is `gate-cmd 2>&1 | tee
/tmp/x.log`, and GitHub's implicit shell for a `run:` step is `bash -e {0}` —
without `pipefail`. The pipeline's status was therefore tee's, always 0.
Observed in intendent run 30638282003: pip-audit reported 10 unignored CVEs
and exited 1, and the job went green. The file header asserted the opposite
("bash runs with -o pipefail"). Declaring `defaults.run.shell: bash` at the
workflow level selects `bash --noprofile --norc -eo pipefail {0}`, so a gate's
exit code is now its step's. Jobs set only `working-directory`, which merges
with this per-key.

**The deps scan is ~200 sequential PyPI lookups.** 6m15s of a 6m30s job in
intendent, where every other gate lands under 2m30s. The verdict is a pure
function of the locked set, the allowlist, and the upstream databases, so a PR
touching none of poetry.lock / pyproject.toml / .colormath/audit.conf can only
differ from its base by the databases — which the base re-checks on every
merge. Those PRs now skip; every non-PR event still scans in full. New
`deps-skip-unchanged` input (default true) opts out.

Every uncertain path in the new step resolves to scan=true.
…d-deps

Resolves the CHANGELOG.md conflict. Both release entries were inserted at
the same point below the header: v3.0.0 (this branch) and v2.4.0 (the
review-ticket skill, merged to main in #10). Kept both in descending
version order — v3.0.0, then v2.4.0, then v2.3.0. No content dropped from
either side.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3dRiaun8tKHmqQWCpMWMo
@craigmbooth
craigmbooth merged commit b268571 into main Aug 1, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant