Skip to content

Update threat detection to download prompt.txt artifact instead of embedding WORKFLOW_MARKDOWN#1946

Merged
pelikhan merged 4 commits intomainfrom
copilot/update-detection-prompt-workflow
Oct 18, 2025
Merged

Update threat detection to download prompt.txt artifact instead of embedding WORKFLOW_MARKDOWN#1946
pelikhan merged 4 commits intomainfrom
copilot/update-detection-prompt-workflow

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Oct 18, 2025

Problem

When agentic workflows have large markdown content, the threat detection job was hitting GitHub Actions' 21KB limit for environment variable values. The WORKFLOW_MARKDOWN environment variable was being used to embed the entire workflow prompt content, which could easily exceed this limit for complex workflows.

Solution

This PR updates the threat detection system to download the prompt.txt artifact instead of embedding the workflow markdown as an environment variable. The prompt.txt artifact is already uploaded by the main agent job, so this change leverages existing infrastructure.

Key Changes

  1. Added prompt artifact download step - The threat detection job now downloads the prompt.txt artifact alongside the existing agent output and patch artifacts.

  2. Updated threat detection template - Modified templates/threat_detection.md to reference the prompt file path ({WORKFLOW_PROMPT_FILE}) instead of embedding the full markdown content ({WORKFLOW_MARKDOWN}). The template now instructs the LLM to load and read the prompt file.

  3. Updated setup script - The JavaScript setup script in buildSetupScript() now checks for the downloaded prompt file and passes the file path information to the LLM instead of embedding the content inline.

  4. Removed WORKFLOW_MARKDOWN environment variable - Eliminated the WORKFLOW_MARKDOWN environment variable from buildWorkflowContextEnvVars(), keeping only WORKFLOW_NAME and WORKFLOW_DESCRIPTION.

  5. Added jqschema import to audit workflow - Added shared/jqschema.md import to the audit-workflows.md workflow to provide JSON schema discovery utilities for analyzing complex tool outputs.

Benefits

  • Avoids size limits: No longer subject to GitHub Actions' 21KB environment variable limit
  • Better performance: Downloading artifacts is more efficient than embedding large content in environment variables
  • Cleaner architecture: Improved separation of concerns between prompt generation and threat detection
  • Backward compatible: Works with all existing workflows without requiring changes to workflow markdown files

Testing

  • Added 3 new unit tests to verify the prompt download mechanism
  • All 64 repository workflows recompiled successfully
  • Verified WORKFLOW_MARKDOWN is no longer present in any compiled workflow
  • All unit tests passing
  • Audit workflow with jqschema import compiles successfully

Example

Before: Threat detection job failed with workflows containing >21KB of markdown content

After: Threat detection job downloads the prompt file and instructs the LLM to load it:

- name: Download prompt artifact
  continue-on-error: true
  uses: actions/download-artifact@v5
  with:
    name: prompt.txt
    path: /tmp/gh-aw/threat-detection/

The threat detection prompt now contains:

## Workflow Source Context

The workflow prompt file is available at: /tmp/gh-aw/threat-detection/prompt.txt (X bytes)

Load and read this file to understand the intent and context of the workflow.

Fixes issue with large workflow prompts exceeding environment variable size limits.

Original prompt

Update the detection creation prompt to avoid WORKFLOW_MARKDOWN.

Instead download the prompt.txt artifact and prompt the LLM to load the file.

This should take care of size issue when the prompt is too large.


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot add import jqschema.md to copilot agent auditor agentic workflow and it should compile correctly

Copilot AI and others added 2 commits October 18, 2025 13:54
…ing WORKFLOW_MARKDOWN

- Add download step for prompt.txt artifact in buildDownloadArtifactStep
- Update threat_detection.md template to reference prompt file path instead of embedding markdown content
- Update buildSetupScript to check for prompt file and use WORKFLOW_PROMPT_FILE placeholder
- Remove WORKFLOW_MARKDOWN environment variable from buildWorkflowContextEnvVars
- All tests passing

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
- Add TestDownloadArtifactStepIncludesPrompt to verify prompt download step
- Add TestSetupScriptReferencesPromptFile to verify setup script uses prompt file
- Add TestBuildWorkflowContextEnvVarsExcludesMarkdown to verify WORKFLOW_MARKDOWN removal
- All tests passing, linting clean

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Update detection creation prompt to avoid WORKFLOW_MARKDOWN Update threat detection to download prompt.txt artifact instead of embedding WORKFLOW_MARKDOWN Oct 18, 2025
Copilot AI requested a review from pelikhan October 18, 2025 14:03
- Added shared/jqschema.md to the imports list in audit-workflows.md
- Workflow compiles successfully with the import
- jqschema utility script and documentation now available to the audit agent

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Oct 18, 2025

@copilot add import jqschema.md to copilot agent auditor agentic workflow and it should compile correctly

Added shared/jqschema.md import to audit-workflows.md and verified it compiles successfully. The jqschema utility script and documentation are now available to the audit agent. Commit: e9e8844

@pelikhan pelikhan marked this pull request as ready for review October 18, 2025 16:09
@github-actions
Copy link
Copy Markdown
Contributor

Agentic Changeset Generator triggered by this pull request.

@pelikhan pelikhan merged commit 2554f15 into main Oct 18, 2025
9 of 10 checks passed
@pelikhan pelikhan deleted the copilot/update-detection-prompt-workflow branch October 18, 2025 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants