threat-detection: workflow context and detection env vars never reach the external detector execution step

[davidslater]

Created by GitHub Ace · View Session

Summary

buildExternalDetectorExecutionStep (pkg/workflow/threat_detection_external.go:392-405) builds its env block exclusively from the engine's own execution step:

executionSteps := engine.GetExecutionSteps(threatDetectionData, constants.ThreatDetectionLogPath)
envLines := extractStepEnvLines(executionSteps[0])

Five env vars that gh-aw already computes are set only on the Setup threat detection github-script step (buildThreatDetectionAnalysisStep) and on the conclude step — never on the step that actually runs the binary.

Verified upstream against a freshly compiled smoke-copilot-standalone.lock.yml (gh-aw v0.84.3):

Step	GH_AW_DETECTION_CONTINUE_ON_ERROR	HAS_PATCH	WORKFLOW_NAME
Setup threat detection (JS)	✅	✅	✅
Execute threat detection with AWF (runs threat-detect)	❌	❌	❌
Conclude threat detection	✅	—	—

The invocation passes no equivalent flags either:

threat-detect --engine copilot --output /tmp/gh-aw/threat-detection/detection_result.json /tmp/gh-aw/threat-detection

Impact

safe-outputs.threat-detection.prompt is silently ignored

CUSTOM_PROMPT is appended verbatim by the detector as ## Additional Instructions. Because it never arrives, a documented, user-facing configuration key has no effect and emits no warning on the external path. Anyone relying on custom detection instructions loses them without knowing.

Every external detection run reports the workflow as unnamed

The prompt renders Unnamed Workflow / No description provided and the runlog records workflow_name_defaulted=true. The detection prompt's own security guidelines instruct the model to use workflow intent to separate legitimate behavior from threats, so removing it should measurably increase false positives.

Strict/warn mode disagreement between stages

GH_AW_DETECTION_CONTINUE_ON_ERROR reaches gh-aw's JS setup step but not the detector. After v0.4.0 (#751) the detector performs its own artifact validation gated on this var, so a workflow configured with threat-detection: continue-on-error: false gets strict validation in the JS setup and lenient validation in the detector. This matters increasingly as the JS setup step is retired. HAS_PATCH is part of the same validation input set.

No regression exists today — absent vars mean the detector sees warn mode, its documented default.

Proposed change

In buildExternalDetectorExecutionStep, append to the execution step's env block:

GH_AW_DETECTION_CONTINUE_ON_ERROR: <same expr already emitted for setup/conclude>
HAS_PATCH: ${{ needs.agent.outputs.has_patch }}
WORKFLOW_NAME: <workflow name>
WORKFLOW_DESCRIPTION: <workflow description>
CUSTOM_PROMPT: <safe-outputs.threat-detection.prompt, when configured>

Reuse the existing c.buildWorkflowContextEnvVars(data) and the conditional CUSTOM_PROMPT: %q logic from buildThreatDetectionAnalysisStep rather than duplicating it.

Flags vs env — recommendation

Upstream #742 added --workflow-name, --workflow-description, --custom-prompt, and --custom-prompt-file specifically so this plumbing cannot be dropped by an env filter. Upstream's stated preference for the custom prompt is --custom-prompt-file, since it survives env-passthrough filtering and avoids shell-quoting an arbitrary user-supplied string into a command line.

Recommendation: use flags for the three prompt-context values (writing the custom prompt to a file staged in the detection dir), and env for GH_AW_DETECTION_CONTINUE_ON_ERROR / HAS_PATCH which are boolean-ish and already env-shaped. Passing user-controlled text as a flag value on a command line assembled by the compiler warrants the file indirection.

If env is chosen instead: the detection step runs awf --env-all, so step-level env reaches the sandboxed process and no AWF change is needed. Still verify the three names survive ComputeAWFExcludeEnvVarNames.

Acceptance criteria

• Decision recorded: flags (--workflow-name / --workflow-description / --custom-prompt-file) vs env.
• Workflow context reaches the detector; runlog prompt_built shows workflow_name_defaulted=false and custom_prompt_applied=true when configured.
• GH_AW_DETECTION_CONTINUE_ON_ERROR and HAS_PATCH set on the execution step.
• Compiler test asserting all five appear on Execute threat detection with AWF.
• Compiler test asserting CUSTOM_PROMPT / --custom-prompt-file is omitted when safe-outputs.threat-detection.prompt is unset.
• make recompile.

References

• Parity: CUSTOM_PROMPT / WORKFLOW_NAME / WORKFLOW_DESCRIPTION never reach threat-detect, so threat-detection.prompt is silently ignored gh-aw-threat-detection#688
• Upstream draft (for github/gh-aw): compiler-side changes needed for external threat-detect parity gh-aw-threat-detection#726 (item B)
• feat(cli): add explicit workflow-context flags and prompt diagnostics gh-aw-threat-detection#742, #751
• gh-aw Honor GH_AW_DETECTION_CONTINUE_ON_ERROR in detection setup to prevent false job failures #49415

[github-actions]

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

Assigning to @pelikhan — matches the pkg/workflow threat detection compiler code area they recently worked in.

Generated by 🔧 Daily Assign Issue To User · auto · 56.7 AIC · ⊞ 5.6K · ◷