Block silent detection pass and classify Claude embedded 429 rate limits

[Copilot]

The detection flow had two coupled failure-mode bugs: Claude CLI embedded 429s were not recognized as rate-limit errors, and detection could still conclude warning/green after a hard CLI failure with no THREAT_DETECTION_RESULT. This change makes 429 classification resilient to stream-json payloads and makes detection fail closed when execution itself failed.

Bug Fix

What was the bug?

• Rate-limit detection relied on limited signals and missed Claude stream-json 429 payloads (api_error_status: 429 / Request rejected (429)), so retry classification was wrong.
• Parse/conclude logic in warn mode could emit warning for parse/agent failures even when the detection execution step had already failed, allowing downstream safe_outputs progression.

How did you fix it?

• Claude harness: broadened 429 detection
  • Expanded isRateLimitError matching to include embedded stream-json 429 indicators and common 429 message variants.
• Detection conclusion: fail-closed on execution failure
  • Wired steps.detection_agentic_execution.outcome into parse/conclude env as DETECTION_AGENTIC_EXECUTION_OUTCOME.
  • In parse_threat_detection_results.cjs, escalate agent_failure / parse_error to hard failure when execution outcome is failure, even if continue-on-error is enabled.
• Generated wrapper catch path parity
  • Applied the same escalation rule in the generated github-script load-error fallback logic.
• Targeted regression coverage
  • Added/updated unit tests for embedded-429 classification, failure escalation behavior, and workflow env wiring.

const detectionExecutionFailed = process.env.DETECTION_AGENTIC_EXECUTION_OUTCOME === "failure";
const mustFail = detectionExecutionFailed && (reason === "agent_failure" || reason === "parse_error");

if (continueOnError && !mustFail) {
  core.setOutput("conclusion", "warning");
} else {
  core.setOutput("conclusion", "failure");
  core.setFailed(message);
}

[Copilot]

Pull request overview

Fixes threat-detection failure classification and “fail-open” behavior by (1) recognizing additional Claude CLI embedded 429/rate-limit signals and (2) ensuring the detection conclusion logic fails closed when the detection execution step itself failed, even if continue-on-error is enabled.

Changes:

• Wire steps.detection_agentic_execution.outcome into the parse/conclude step env and use it to force failure on execution-failure + parse/agent failures.
• Expand Claude harness rate-limit matching to catch stream-json embedded 429 indicators and common 429 message variants.
• Add targeted regression tests for the new env wiring and failure-escalation behavior.

Show a summary per file

File	Description
pkg/workflow/threat_detection.go	Passes execution outcome into parse/conclude env and updates generated github-script fallback to fail when execution failed.
pkg/workflow/detection_success_test.go	Verifies compiled workflow includes the new DETECTION_AGENTIC_EXECUTION_OUTCOME env var.
actions/setup/js/parse_threat_detection_results.cjs	Escalates parse/agent failures to hard failure when execution outcome indicates failure.
actions/setup/js/parse_threat_detection_results.test.cjs	Adds coverage for “execution failed => must fail even in warn mode”.
actions/setup/js/claude_harness.cjs	Broadens 429/rate-limit detection regex used by the Claude CLI harness.
actions/setup/js/claude_harness.test.cjs	Adds unit tests for newly recognized embedded-429 signals.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

actions/setup/js/parse_threat_detection_results.cjs:285

• setDetectionFailure() uses warn-mode behavior for all reasons except the new mustFail cases. Because the downstream safe_outputs gate checks needs.detection.result == 'success' (not the success output), calling setDetectionFailure('threat_detected', ...) in warn mode will not fail the step/job and can allow safe outputs to proceed even when threats are detected. Treat reason === 'threat_detected' as always-fatal (or otherwise ensure the detection job result is failure when threats are present), independent of continue-on-error.

  function setDetectionFailure(reason, message) {
    const mustFail = detectionExecutionFailed && (reason === "agent_failure" || reason === "parse_error");
    core.setOutput("reason", reason);
    if (isWarnMode && !mustFail) {
      core.warning(`⚠️ ${message}`);
      core.setOutput("conclusion", "warning");
      core.setOutput("success", "false");
    } else {
      core.setOutput("conclusion", "failure");

• Files reviewed: 6/6 changed files
• Comments generated: 0