Skip to content

ci(wave3): stub-sentinel detection step in tier3-container matrix#157

Merged
hyperpolymath merged 1 commit into
mainfrom
chore/wave3-stub-sentinel-detection
May 30, 2026
Merged

ci(wave3): stub-sentinel detection step in tier3-container matrix#157
hyperpolymath merged 1 commit into
mainfrom
chore/wave3-stub-sentinel-detection

Conversation

@hyperpolymath
Copy link
Copy Markdown
Owner

Summary

Second half of #75 — the per-prover smoke-check step in the weekly Tier-3 container build uses `|| true` for backends whose CLIs exit non-zero on quit (metamath/hol4/acl2/etc.). That was correct for the per-backend version-check intent, but it also silently swallowed the case where a graceful stub fallback (defined in `Containerfile.wave3` for tamarin/proverif/scip/metamath) printed its sentinel and exited 1. Result: a broken upstream pin masquerading as a passing weekly build — exactly the regression #75 was filed to surface.

This adds a new step (`Stub-sentinel detection (#75)`) that re-runs each matrix cell's `version_check`, captures stdout+stderr verbatim, and fails LOUDLY with a `::error::` annotation on any of:

Pattern Source backend
`not available (bundle install failed` tamarin, scip
`not available (build failed` metamath
`not available (source build failed` proverif
`bundle install failed at image build time` generic
`build failed at image build time` generic
`source build failed at image build` generic

Out of scope (by design)

  • Imandra is not matched — its stub says "proprietary licence required" rather than the build-time sentinels, and it's excluded from the matrix anyway (not licensable in public CI).
  • Hard-fail backends without a graceful stub (or-tools, hol4, acl2, poly/ml) — these crash the build step itself if their upstream is dead, so they're already covered.

Gate semantics

`continue-on-error: true` on the job is preserved. These weekly images are informational, not merge gates — so a stub detection shows as red ❌ inside an amber/yellow job, visible in the Actions UI without blocking. That matches the rest of the `tier3-container` contract.

Companion

Bumps for the stale pins (Poly/ML / OR-Tools / ACL2) shipped in #156. With both this PR and #156 merged, #75's two asks are addressed:

  1. ✓ Bump stale pins.
  2. ✓ Catch silent stub-degradation in CI before it lingers another month.

Test plan

  • Workflow lint / actionlint passes.
  • Next weekly Tier-3 cron run shows the new step appearing in each matrix cell.
  • Manual probe: if a future pin breaks (or someone simulates it by replacing an upstream URL with a 404 in a fork), the stub-sentinel step fires red.

Refs #75.

🤖 Generated with Claude Code

Second half of #75: the per-prover smoke-check step uses \`|| true\`
for backends whose CLIs exit non-zero on quit (metamath/hol4/acl2/etc.)
This was correct for those backends in isolation, but it also swallowed
the case where a graceful stub fallback (defined in Containerfile.wave3
for tamarin/proverif/scip/metamath) printed its sentinel line —
\"<prover> not available (... failed at image build time)\" — and
exited 1. Result: a broken upstream pin masked as a passing weekly
container build, exactly the regression #75 was filed to surface.

This step re-runs the matrix's \`version_check\`, captures stdout+stderr
verbatim, and fails LOUDLY (\`::error::\` annotation + exit 1) on any
of the known sentinel patterns:

  - \"not available (bundle install failed\"     (tamarin, scip)
  - \"not available (build failed\"              (metamath)
  - \"not available (source build failed\"       (proverif)
  - \"bundle install failed at image build time\" (generic)
  - \"build failed at image build time\"          (generic)
  - \"source build failed at image build\"        (generic)

Imandra is intentionally NOT pattern-matched here — its stub message
includes \"proprietary licence required\" rather than the build-time
sentinels, and it is excluded from the matrix anyway (not licensable
in public CI). Hard-fail backends without a graceful stub (or-tools,
hol4, acl2, poly/ml) are covered by the existing build step failing
the cell outright.

\`continue-on-error: true\` on the job is preserved — these weekly
images are informational, not merge gates — so a stub detection shows
as a red ❌ step inside an amber/yellow job, visible in the Actions
UI without blocking. That matches the rest of the tier3-container
contract.

Refs #75 (closes the second half — the bumps shipped in #156).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@hyperpolymath hyperpolymath enabled auto-merge (squash) May 30, 2026 22:10
@hyperpolymath hyperpolymath merged commit e86d156 into main May 30, 2026
32 of 40 checks passed
@hyperpolymath hyperpolymath deleted the chore/wave3-stub-sentinel-detection branch May 30, 2026 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant