Skip to content

fix(conformance): fix WTD-001/WTD-002 false-positives on centralized threat-detection label string - #49753

Merged
pelikhan merged 2 commits into
mainfrom
copilot/wtd-001-fix-footer-generator-error
Aug 2, 2026
Merged

fix(conformance): fix WTD-001/WTD-002 false-positives on centralized threat-detection label string#49753
pelikhan merged 2 commits into
mainfrom
copilot/wtd-001-fix-footer-generator-error

Conversation

Copilot AI commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

The conformance checker incorrectly reports CRITICAL/HIGH failures for WTD-001 (req 2) and WTD-002 because it greps handler .cjs files for the literal string "agentic threat detected", which is never verbatim in those files — it lives in the centralized threat_detection_warning.cjs helper and threat_detection_caution.md template, reached via delegation.

Changes

scripts/check-safe-outputs-conformance.sh

  • WTD-001 req 2 (check_wtd_reviewable_annotation): replaces the direct grep on generate_footer.cjs with a fallback chain matching the pattern already used for req 3 (the XML marker check): if the string isn't literal in the footer, accept it when the footer references threat_detection_caution via renderTemplateFromFile and the string is present in threat_detection_caution.md or threat_detection_warning.cjs.

  • WTD-002 (check_wtd_convertible_fallback): adds an equivalent fallback — if "agentic threat detected" isn't literal in push_to_pull_request_branch.cjs, accept it when the handler calls getThreatWarningPresentation and the string is present in threat_detection_warning.cjs.

# Before (WTD-001 req 2)
if ! grep -q "agentic threat detected" "$footer_file"; then
    log_critical "..."
fi

# After — mirrors req 3's existing fallback pattern
local label_found=0
if grep -q "agentic threat detected" "$footer_file" 2>/dev/null; then
    label_found=1
elif grep -q "threat_detection_caution" "$footer_file" 2>/dev/null; then
    local caution_template="actions/setup/md/threat_detection_caution.md"
    if [ -f "$caution_template" ] && grep -q "agentic threat detected" "$caution_template" 2>/dev/null; then
        label_found=1
    elif [ -f "$threat_warning_file" ] && grep -q "agentic threat detected" "$threat_warning_file" 2>/dev/null; then
        label_found=1
    fi
fi

No changes to handler source files — the centralized-template approach is intentional.

…threat-detection label string

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix false positives in conformance checker for label string fix(conformance): fix WTD-001/WTD-002 false-positives on centralized threat-detection label string Aug 2, 2026
Copilot AI requested a review from pelikhan August 2, 2026 10:51
@pelikhan
pelikhan marked this pull request as ready for review August 2, 2026 11:16
Copilot AI review requested due to automatic review settings August 2, 2026 11:16
@pelikhan
pelikhan merged commit c1c1017 into main Aug 2, 2026
@pelikhan
pelikhan deleted the copilot/wtd-001-fix-footer-generator-error branch August 2, 2026 11:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes false positives in WTD-001 and WTD-002 conformance checks by recognizing centralized threat-warning sources.

Changes:

  • Adds template/helper fallback validation for WTD-001.
  • Adds helper delegation validation for WTD-002.
Show a summary per file
File Description
scripts/check-safe-outputs-conformance.sh Recognizes centralized threat-detection label generation.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 1/1 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment on lines +1353 to +1357
if [ -f "$caution_template" ] && grep -q "agentic threat detected" "$caution_template" 2>/dev/null; then
label_found=1
elif [ -f "$threat_warning_file" ] && grep -q "agentic threat detected" "$threat_warning_file" 2>/dev/null; then
label_found=1
fi
Comment on lines +1421 to +1423
elif grep -q "getThreatWarningPresentation" "$push_handler" 2>/dev/null; then
local threat_warning_file="actions/setup/js/threat_detection_warning.cjs"
if [ -f "$threat_warning_file" ] && grep -q "agentic threat detected" "$threat_warning_file" 2>/dev/null; then
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.84.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Safe Outputs Conformance] WTD-001/WTD-002: Conformance checker false-positives on centralized threat-detection label string

3 participants