fix(strix): retry artifact-less provider crashes without weakening exhaustion

[seonghobae]

Problem

A provider crash can emit fatal/warning text and severity-like console markers without producing a vulnerabilities/*.md report. The gate previously interpreted that crash debris as unmapped vulnerability evidence and returned non-recoverable before trying configured fallback models. Separately, the workflow used to convert exhausted provider failures into a neutral success, leaving the required security context green without completed evidence.

Observed on wardnet #68: the primary NVIDIA NIM model returned HTTP 429, produced no vulnerability artifact, and the fallback chain never ran.

Changes

Retry only the artifact-less infrastructure classification

• snapshot report run directories before each model attempt;
• scope report failure-signal attribution to directories created by that attempt, so a crashed primary cannot poison a clean fallback success;
• classify INFRA_ERROR_DETECTED with no current-run vulnerabilities/*.md artifact as retryable across fallback models;
• preserve every actual vulnerability report artifact across attempts and fail before accepting a fallback success when a threshold finding exists.

Add bounded same-provider fallbacks

For nvidia_nim mode, try the existing approved Nemotron candidates before degrading to GitHub Models:

1. nvidia_nim/nvidia/llama-3.1-nemotron-ultra-253b-v1
2. nvidia_nim/nvidia/nemotron-3-super-120b-a12b
3. github_models/openai/o3
4. github_models/openai/gpt-5-chat

No new credentials or unbounded retry path are introduced.

Preserve fail-closed exhaustion

After all configured attempts, any nonzero Strix result remains nonzero. Provider unavailability without a vulnerability report is incomplete security evidence; it is not converted into a neutral success. Artifacts and the exact gate console log are still collected under always() for diagnosis.

Regression contracts

• artifact-less 429 crash with log-only severity debris → fallback is attempted and a clean fallback may pass;
• 429 crash with a real CRITICAL report artifact → fails before fallback;
• later attempts ignore earlier run-directory warning/fatal logs while all vulnerability artifacts remain globally visible;
• wrapper propagates the exact nonzero gate exit under the same Bash flags used by Actions;
• quota exhaustion with no report remains a failed required check;
• host environments without Bash skip only the executable wrapper contracts;
• the bounded required-path smoke test pins both retry classification and final fail-closed exhaustion.

Scope

Seven files change: the Strix workflow, gate, full and bounded shell contracts, shared Bash-test collection guard, required-workflow contract, and executable workflow-wrapper contract.

Supersession

This PR supersedes #733's fallback-only change and #587's wrapper-only fail-closed change by composing both with the crash-debris retry classifier. It deliberately does not synthesize success or weaken the Medium+ vulnerability threshold.

[coderabbitai]

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 3 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c5c1dc72-ddd3-4e9f-bcac-d0e89abcb454

📥 Commits

Reviewing files that changed from the base of the PR and between 3f65dbe and 269456c.

📒 Files selected for processing (7)

• .github/workflows/strix.yml
• scripts/ci/strix_quick_gate.sh
• scripts/ci/strix_required_workflow_smoke.sh
• scripts/ci/test_strix_quick_gate.sh
• tests/conftest.py
• tests/test_required_workflow_queue_contract.py
• tests/test_strix_workflow_shell_contract.py

---

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[opencode-agent]

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

• Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

• Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

• Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

• Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

• Result: REQUEST_CHANGES

• Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 269456cfe749385f85e1928f3ebd30252d556504.

• Head SHA: 269456cfe749385f85e1928f3ebd30252d556504

• Workflow run: 30900198092

• Workflow attempt: 1

Coverage evidence

Coverage Decision

• Result: FAIL
• Test evidence: not proven passing
• Docstring evidence: not proven passing when configured
• Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: strix.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: strix.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["CI script (3 files)"]
  S2 --> I2["review and security gate shell path"]
  I2 --> R2["Review risk: CI script (3 files)"]
  R2 --> V2["bash -n plus Strix self-test"]
  Evidence --> S3["Test (3 files)"]
  S3 --> I3["regression suite"]
  I3 --> R3["Review risk: Test (3 files)"]
  R3 --> V3["targeted test run"]

Loading

[opencode-agent]

OpenCode Review Overview

• Head SHA: 269456cfe749385f85e1928f3ebd30252d556504
• Workflow run: 30900198092
• Workflow attempt: 1
• Gate result: REQUEST_CHANGES (approval step)

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

• Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

• Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

• Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

• Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

• Result: REQUEST_CHANGES

• Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 269456cfe749385f85e1928f3ebd30252d556504.

• Head SHA: 269456cfe749385f85e1928f3ebd30252d556504

• Workflow run: 30900198092

• Workflow attempt: 1

Coverage evidence

Coverage Decision

• Result: FAIL
• Test evidence: not proven passing
• Docstring evidence: not proven passing when configured
• Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: strix.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: strix.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["CI script (3 files)"]
  S2 --> I2["review and security gate shell path"]
  I2 --> R2["Review risk: CI script (3 files)"]
  R2 --> V2["bash -n plus Strix self-test"]
  Evidence --> S3["Test (3 files)"]
  S3 --> I3["regression suite"]
  I3 --> R3["Review risk: Test (3 files)"]
  R3 --> V3["targeted test run"]

Loading

[seonghobae]

@opencode-agent Please re-review exact head 269456cfe749385f85e1928f3ebd30252d556504 after central baseline PR #731 lands and exact-head coverage evidence reruns. Verify artifact-less infrastructure crashes are retryable only across the bounded model chain, real vulnerability artifacts remain fail-closed, and provider exhaustion never becomes success. Submit APPROVE only if all current-head evidence passes.

[seonghobae]

@cwl-noema-review Please independently review exact head 269456cfe749385f85e1928f3ebd30252d556504 after #731 repairs the central baseline and current-head checks complete. Confirm bounded retry and fail-closed exhaustion semantics; submit APPROVE only if no blocker remains.