Skip to content

Harden Microsoft Defender scan stage to prevent silent gh-aw.exe scan skips#35494

Merged
pelikhan merged 11 commits into
mainfrom
copilot/review-defender-scanning-logs
May 28, 2026
Merged

Harden Microsoft Defender scan stage to prevent silent gh-aw.exe scan skips#35494
pelikhan merged 11 commits into
mainfrom
copilot/review-defender-scanning-logs

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 28, 2026

MpCmdRun.exe can exit 0 even when the executable was not actually scanned, so relying on $LASTEXITCODE alone can let an unscanned binary pass CI silently.

Changes

  • Fail on skipped/declined scans — captures MpCmdRun output and fails on skip-like messages (case-insensitive), including was skipped, cannot be scanned, not performed, and exclusion-related output
  • Fail on untrusted successful exits — treats exit code 0 as insufficient unless output confirms scan execution (Scan starting and Scan finished)
  • Fail on threat indicators — fails if output contains Threat
  • Harden Defender diagnostics — logs Defender status/signature/version fields from Get-MpComputerStatus, plus additional context from Get-MpPreference (exclusions) and WinDefend service state; all wrapped safely so diagnostics don’t block scan execution
  • Validate and stabilize scan target — verifies gh-aw.exe exists as a file, waits briefly, and confirms size/timestamp stability before scanning
  • Scan from dedicated temp location — copies the executable to a GUID-named file under a dedicated scan directory in temp (instead of scanning directly from the workspace path)
  • Validate copy integrity — compares source/copy size and SHA256 hash before invoking Defender
  • Use explicit Defender scan command — runs MpCmdRun.exe -Scan -ScanType 3 -File <copied file> -DisableRemediation
  • Surface full scan output — prints all stdout/stderr lines and exit code for actionable diagnostics
  • Improve diagnostic readability — normalizes output to strings and ensures single-line skip matches are joined correctly
$workspaceBinaryPath = Join-Path $PWD "gh-aw.exe"
if (-not (Test-Path -Path $workspaceBinaryPath -PathType Leaf)) {
  Write-Error "Built binary not found: $workspaceBinaryPath"
  exit 1
}

# Stabilize and hash source
$initial = Get-Item -LiteralPath $workspaceBinaryPath
Start-Sleep -Seconds 3
$stable = Get-Item -LiteralPath $workspaceBinaryPath
if ($initial.Length -ne $stable.Length -or $initial.LastWriteTimeUtc -ne $stable.LastWriteTimeUtc) { exit 1 }
$sourceHash = (Get-FileHash -LiteralPath $workspaceBinaryPath -Algorithm SHA256).Hash

# Copy to temp scan path and verify integrity
$scanBasePath = Join-Path $env:TEMP "defender-scan"
New-Item -ItemType Directory -Path $scanBasePath -Force | Out-Null
$binaryPath = Join-Path $scanBasePath "scan-$([guid]::NewGuid().ToString('N')).exe"
Copy-Item -LiteralPath $workspaceBinaryPath -Destination $binaryPath -Force
$copyHash = (Get-FileHash -LiteralPath $binaryPath -Algorithm SHA256).Hash
if ($copyHash -ne $sourceHash) { exit 1 }

$output = & $mpCmdRun -Scan -ScanType 3 -File $binaryPath -DisableRemediation 2>&1 | ForEach-Object { "$_" }
$scanExitCode = $LASTEXITCODE
$outputText = @($output) -join "`n"

$skipped = $output | Where-Object { $_ -imatch "\bwas skipped\b|\bcannot be scanned\b|\bnot performed\b|\b(?:file|scan).*\bexcluded\b" }
$threatLines = $output | Where-Object { $_ -match "\bThreat\b" }
$scanStarted = $outputText -imatch "\bScan starting\b"
$scanFinished = $outputText -imatch "\bScan finished\b"

if ($scanExitCode -ne 0 -or $skipped -or $threatLines -or -not ($scanStarted -and $scanFinished)) {
  exit 1
}

Copilot AI and others added 3 commits May 28, 2026 14:51
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…ailure

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan pelikhan marked this pull request as ready for review May 28, 2026 15:20
Copilot AI review requested due to automatic review settings May 28, 2026 15:20
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 28, 2026

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 28, 2026

PR Code Quality Reviewer completed the code quality review.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 28, 2026

Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR #35494 does not have the 'implementation' label and has 0 new lines of code in default business logic directories (≤100 threshold not exceeded).

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 28, 2026

🧪 Test Quality Sentinel completed test quality analysis.

No test files were added or modified in this PR. Test Quality Sentinel skipped.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates CI workflows to prevent Windows Microsoft Defender scans from silently passing when scanning is skipped, and adds an extra fetch step for the cross-repo smoke workflow to bring in additional refs.

Changes:

  • Capture MpCmdRun.exe output/exit code, fail the job if Defender reports the file “was skipped”, and print full scan output for debugging.
  • Emit Microsoft Defender status/signature/version diagnostics before scanning.
  • Fetch additional main + PR head refs in the cross-repo smoke update workflow.
Show a summary per file
File Description
.github/workflows/smoke-update-cross-repo-pr.lock.yml Adds an authenticated git fetch step to pull in additional refs for the side repo workflow execution.
.github/workflows/cgo.yml Hardens the Windows Defender scanning step by logging diagnostics, surfacing full MpCmdRun output, and failing on “scan skipped” despite exit code 0.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 6

Comment thread .github/workflows/cgo.yml Outdated
Comment thread .github/workflows/smoke-update-cross-repo-pr.lock.yml
Comment on lines +1683 to +1687
env:
GH_AW_FETCH_TOKEN: ${{ secrets.GH_AW_SIDE_REPO_PAT }}
run: |
header=$(printf "x-access-token:%s" "${GH_AW_FETCH_TOKEN}" | base64 -w 0)
git -c "http.extraheader=Authorization: Basic ${header}" fetch origin '+refs/heads/main:refs/remotes/origin/main' '+refs/pull/*/head:refs/remotes/origin/pull/*/head'
Comment thread .github/workflows/cgo.yml Outdated
Comment on lines +1681 to +1687
- name: Fetch additional refs for github/gh-aw-side-repo
if: (!cancelled()) && needs.agent.result != 'skipped' && contains(needs.agent.outputs.output_types, 'push_to_pull_request_branch')
env:
GH_AW_FETCH_TOKEN: ${{ secrets.GH_AW_SIDE_REPO_PAT }}
run: |
header=$(printf "x-access-token:%s" "${GH_AW_FETCH_TOKEN}" | base64 -w 0)
git -c "http.extraheader=Authorization: Basic ${header}" fetch origin '+refs/heads/main:refs/remotes/origin/main' '+refs/pull/*/head:refs/remotes/origin/pull/*/head'
Comment thread .github/workflows/cgo.yml Outdated
@github-actions github-actions Bot mentioned this pull request May 28, 2026
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔎 Code quality review by PR Code Quality Reviewer · sonnet46 1.2M

- name: Fetch additional refs for github/gh-aw-side-repo
if: (!cancelled()) && needs.agent.result != 'skipped' && contains(needs.agent.outputs.output_types, 'push_to_pull_request_branch')
env:
GH_AW_FETCH_TOKEN: ${{ secrets.GH_AW_SIDE_REPO_PAT }}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Derived credential value won't be masked in Actions logs: the header variable holds a base64-encoded form of the PAT that GitHub Actions never registers as a secret — only the raw GH_AW_FETCH_TOKEN value is masked.

💡 Details and suggested fix

GitHub Actions masks secrets by their literal value. base64 produces a different string, so $header is unmasked. If ACTIONS_STEP_DEBUG=true is set on this runner (e.g. for routine debugging), or if any downstream shell error prints the git config, the PAT is exposed in cleartext-equivalent form.

A safer pattern is to pass the credential directly via git -c credential.helper= or via a GITHUB_TOKEN-style env var that git reads natively, without materialising a base64 derivative in a shell variable:

# Option A: pass via git credential helper (never surfaces in shell)
git -c "credential.helper=" \
    -c "(github.com.username/redacted) \
    -c "(github.com.password/redacted) \
    fetch origin ...

# Option B: add a ::add-mask:: annotation to hide derived value
printf '%s' "$(printf 'x-access-token:%s' "${GH_AW_FETCH_TOKEN}" | base64 -w 0)" >> $GITHUB_ENV
# then ::add-mask:: the env var before using it

Alternatively, if the value is known at step start, emit echo "::add-mask::$(printf ... | base64 -w 0)" before using $header.

Comment thread .github/workflows/cgo.yml
# Use -imatch for case-insensitive matching since the exact casing of
# MpCmdRun output may vary across Defender versions.
$skipped = $output | Where-Object { $_ -imatch "was skipped" }
if ($skipped) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skip detection is purely negative and will silently regress if Defender changes its output: the check only looks for "was skipped" and treats its absence as a successful scan — but absence of a skip string is not the same as presence of a success string.

💡 Details and suggested fix

If a future Defender update changes the skip message wording (e.g. "scan was not performed", "file excluded", or the message disappears entirely), this guard silently stops working and CI goes back to passing on unskipped scans — the exact regression this PR exists to fix.

A more robust approach requires the output to contain a positive confirmation before declaring success:

# Require a positive success indicator as well
$succeeded = $output | Where-Object { $_ -imatch "Threat scan completed" -or $_ -imatch "No threats detected" }
if (-not $succeeded) {
  Write-Error "MpCmdRun did not emit a known success message — cannot confirm scan actually ran."
  Write-Error "Full output: $($output -join "`n")"
  exit 1
}

At minimum, document clearly in CI runbooks which exact Defender version was tested so a version upgrade triggers a manual verification.

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /diagnose — no blocking issues; two suggestions to harden the fix further.

📋 Key Themes & Highlights

Key Themes

  • Pattern coverage: only "was skipped" is checked; Defender can silently bail for other reasons that would still be missed
  • ErrorRecord coercion: 2>&1 delivers stderr as ErrorRecord objects; an explicit "$_" cast would make the code unambiguous

Positive Highlights

  • ✅ Root cause correctly identified and addressed — $LASTEXITCODE alone is insufficient for Defender
  • ✅ Capturing 2>&1 and printing all output before checking is excellent for future debuggability
  • Get-MpComputerStatus diagnostics block will make flaky-scan incidents much easier to investigate
  • -imatch case-insensitive matching is the right choice for cross-version output compatibility
  • ✅ Informative Write-Error messages with actionable "Possible causes" guidance

Note on lock file change

The smoke-update-cross-repo-pr.lock.yml fetch-refs addition appears unrelated to the Defender fix — worth confirming it was intentional to bundle here.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 1.2M

Comment thread .github/workflows/cgo.yml Outdated
Comment thread .github/workflows/cgo.yml Outdated
@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot review all comments from copilot and github-actions

Copilot AI and others added 2 commits May 28, 2026 16:05
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 28, 2026

@copilot review all comments from copilot and github-actions

Addressed the actionable Copilot/GitHub Actions feedback in commits e7b019c and 8eb6f9c.

Key updates:

  • Wrapped Get-MpComputerStatus in try/catch and renamed variable to $mpStatus
  • Normalized MpCmdRun output to strings before matching/logging
  • Expanded and tightened skip-pattern detection with safer regex boundaries
  • Fixed skipped-output join behavior for single-line matches (@($skipped) -join '; ')

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot the binary path looks incorrect. Add a check that the binary exists before running the scanner.

https://github.com/github/gh-aw/actions/runs/26587916933/job/78338824734?pr=35494

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 28, 2026

@copilot the binary path looks incorrect. Add a check that the binary exists before running the scanner.

https://github.com/github/gh-aw/actions/runs/26587916933/job/78338824734?pr=35494
...

Implemented in ecfb974: added a stricter pre-scan existence check for gh-aw.exe before Defender runs (Test-Path ... -PathType Leaf), plus directory diagnostics when missing, and path resolution before invoking MpCmdRun.

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot

Harden Microsoft Defender Scanning in GitHub Actions

Implement a robust Windows Defender scanning stage for GitHub Actions Windows runners.

Problem

Current behavior:

Scan starting...
Scan finished.
Scanning <file> was skipped.
Exit code: 0

This is a failure condition.

Exit code: 0 does NOT guarantee the executable was scanned.

The workflow must explicitly detect skipped scans and fail the job.


Requirements

Environment Assumptions

The workflow runs on:

  • GitHub-hosted Windows runners
  • ephemeral workspace paths such as:
D:\a\<repo>\<repo>\<binary>.exe

Potential issues include:

  • Defender passive mode
  • disabled real-time protection
  • CI workspace exclusions
  • file locking/race conditions
  • freshly linked executables
  • transient filesystem behavior

Implementation Requirements

1. Defender Diagnostics

Collect and print:

Get-MpComputerStatus
Get-MpPreference

Include at minimum:

  • AMRunningMode
  • AntivirusEnabled
  • RealTimeProtectionEnabled
  • IoavProtectionEnabled
  • AntivirusSignatureVersion
  • AntivirusSignatureLastUpdated
  • AMProductVersion
  • AMEngineVersion

Also print:

  • Defender exclusions
  • WinDefend service state
  • exact MpCmdRun.exe path
  • executable path
  • executable size
  • SHA256 hash

2. File Stabilization

Before scanning:

  • verify the executable exists
  • wait several seconds
  • verify size and timestamp remain stable

Fail if the file changes during stabilization.


3. Avoid Scanning Workspace Paths

DO NOT scan directly from:

D:\a\

Instead:

  • create a dedicated scan directory:
$env:TEMP\defender-scan

or:

C:\Temp\
  • copy the executable there
  • use a GUID-based filename

Example:

scan-<guid>.exe

4. Validate Copy Integrity

After copying:

  • compare file sizes
  • compare SHA256 hashes

Fail immediately if mismatched.


5. Defender Scan Execution

Use:

MpCmdRun.exe -Scan -ScanType 3 -File <copied file> -DisableRemediation

Capture:

  • stdout
  • stderr
  • exit code

Print all output.


Failure Conditions

The workflow MUST fail if:

Output contains:

was skipped

or:

Threat

or:

  • exit code != 0
  • scan completion cannot be confirmed

Critical Constraint

DO NOT trust:

Exit code: 0

as proof that scanning occurred.

The implementation MUST explicitly parse Defender output to confirm the executable was actually scanned.


Copilot AI and others added 2 commits May 28, 2026 17:46
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 28, 2026

@copilot

Harden Microsoft Defender Scanning in GitHub Actions

...

Implemented in e90b030 (with core hardening in 8a18488): Defender diagnostics now include status/preferences/service/exclusions, the executable is stabilized then copied to a dedicated temp scan path with size+SHA256 integrity checks, scan runs with -DisableRemediation, and the job now fails on skipped scans, threat indicators, non-zero exit, or missing scan-start/scan-finish confirmation.

Copilot AI changed the title Fix silent pass when Microsoft Defender skips gh-aw.exe scan Harden Microsoft Defender scan stage to prevent silent gh-aw.exe scan skips May 28, 2026
@pelikhan pelikhan merged commit 93e67cb into main May 28, 2026
23 checks passed
@pelikhan pelikhan deleted the copilot/review-defender-scanning-logs branch May 28, 2026 18:06
Copilot AI added a commit that referenced this pull request May 28, 2026
…recompile

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
pelikhan added a commit that referenced this pull request May 28, 2026
* Add defender job to release.md; sync_actions now depends on defender

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

* Recompile workflows: regenerate release.lock.yml with defender job

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

* Update defender job: reuse hardened script from cgo.yml (PR #35494), recompile

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: Peli de Halleux <pelikhan@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants