threat-detection: strict-mode conclude failure writes no outputs, rendering infra failures as detected threats

[davidslater]

Created by GitHub Ace · View Session

Summary

actions/setup/sh/conclude_threat_detection.sh exits non-zero on a strict-mode infrastructure failure without writing any step outputs, which makes an infra failure render to reviewers as [!CAUTION] agentic threat detected. This is exactly the confusion #48940 set out to eliminate; the fix landed in the .cjs path and never covered the shell path.

This is a live bug, independent of the external-detector workstream. It needs no new binary and no coordination with gh-aw-threat-detection.

Current code

actions/setup/sh/conclude_threat_detection.sh lines 35-43:

if [ "${continue_on_error}" = "true" ]; then
  echo "::warning::${result_message}; continuing because GH_AW_DETECTION_CONTINUE_ON_ERROR=true"
  echo "conclusion=warning"   >> "${GITHUB_OUTPUT}"
  echo "success=false"        >> "${GITHUB_OUTPUT}"
  echo "reason=agent_failure" >> "${GITHUB_OUTPUT}"
  exit 0
fi
echo "ERR_SYSTEM: ❌ ${result_message}"
exit 1

Failure chain

1. Strict mode (GH_AW_DETECTION_CONTINUE_ON_ERROR=false) + missing detection_result.json → exit 1 with nothing appended to $GITHUB_OUTPUT.
2. needs.detection.outputs.detection_reason is the empty string.
3. In actions/setup/js/threat_detection_warning.cjs, isToolingFailureReason("") → false, so the engine-error branch from fix: use [!WARNING] banner for threat detection engine failures instead of [!CAUTION] #48940 is not taken.
4. getDetectionReasonText("") falls through to the generic "The threat detection analysis could not be completed."
5. Net result: a strict-mode infra failure renders as [!CAUTION] agentic threat detected.

Proposed fix

Write the outputs before failing:

echo "ERR_SYSTEM: ❌ ${result_message}"
echo "conclusion=failure"    >> "${GITHUB_OUTPUT}"
echo "success=false"         >> "${GITHUB_OUTPUT}"
echo "reason=agent_failure"  >> "${GITHUB_OUTPUT}"
exit 1

While in there: the warn-mode branch writes step outputs but does not export GH_AW_DETECTION_CONCLUSION / GH_AW_DETECTION_REASON to $GITHUB_ENV, unlike setDetectionFailure in parse_threat_detection_results.cjs. That is inconsistent for any later step inside the detection job. Export them in both branches.

Acceptance criteria

• Strict-mode missing-result-file path writes conclusion=failure, success=false, reason=agent_failure to $GITHUB_OUTPUT before exit 1.
• Both the warn and strict branches export GH_AW_DETECTION_CONCLUSION / GH_AW_DETECTION_REASON to $GITHUB_ENV.
• Test coverage in pkg/workflow/threat_detection_conclude_script_test.go asserting the outputs in strict mode.
• make recompile.

Relationship to the delegation issue

The follow-on work in this workstream deletes these branches entirely and delegates to threat-detect conclude. This issue should still land now rather than waiting — the delegation change carries more risk and depends on the v0.4.0 pin, and per the upstream note the bug is worth fixing on its own timeline.

References

• Parity: conclusion logic is duplicated in conclude_threat_detection.sh and threat-detect conclude, with divergent (and partly broken) semantics gh-aw-threat-detection#694
• Upstream draft (for github/gh-aw): compiler-side changes needed for external threat-detect parity gh-aw-threat-detection#726 (item E)
• gh-aw fix: use [!WARNING] banner for threat detection engine failures instead of [!CAUTION] #48940

[github-actions]

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

🍪 Issue Monster selected this for Copilot

I have identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · gpt54 · 1.7 AIC · ⊞ 9.3K · ◷

[github-actions]

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

🍪 Issue Monster selected this for Copilot

I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · gpt54 · 2.54 AIC · ⊞ 9.3K · ◷

[github-actions]

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

🍪 Issue Monster selected this for Copilot

I have identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · gpt54 · 5.4 AIC · ⊞ 10.3K · ◷