Skip to content

Add strict mode enforcement for zizmor security findings#2652

Merged
pelikhan merged 6 commits into
mainfrom
copilot/update-compiler-strict-mode
Oct 28, 2025
Merged

Add strict mode enforcement for zizmor security findings#2652
pelikhan merged 6 commits into
mainfrom
copilot/update-compiler-strict-mode

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Oct 28, 2025

Plan: Update strict mode to treat zizmor warnings as errors ✅

Problem Statement

Update "strict" mode in compiler such that zizmor warnings/errors are not allowed and are treated as errors (if zizmor is enabled). Run recompile and report current warnings status.

Implementation Complete ✅

All changes implemented and validated:

  • Strict mode now treats zizmor findings as compilation errors
  • Tests pass
  • Workflows recompiled successfully
  • Merged main branch
  • Fixed test workflows after github.actor removal from allowed expressions

Latest Update: Merge main and fix test workflows

Merged changes from main:

  • Updated allowed expressions (removed github.actor and github.workflow)
  • Updated instruction files
  • Updated workflows to remove deprecated expressions

Fixed test workflows:

  • pkg/cli/workflows/test-all.md - Removed github.actor references
  • pkg/cli/workflows/test-template-github-actions-syntax.md - Removed github.actor and github.workflow
  • pkg/cli/workflows/test-template-issue-context.md - Removed github.actor
  • pkg/cli/workflows/test-template-pr-context.md - Removed github.actor

Compilation Results:

  • Main workflows: 56/56 compiled successfully ✅
  • Test workflows: 65/65 compiled successfully ✅
  • All tests passing ✅

Changes Made

  1. Modified pkg/cli/zizmor.go:

    • Updated runZizmor() to accept a strict bool parameter
    • Modified error handling: when strict mode is enabled, exit codes 10-14 (findings) now return an error
    • Updated parseAndDisplayZizmorOutput() to return the total warning count as int
    • Error message in strict mode: "strict mode: zizmor found X security warnings/errors - workflows must have no zizmor findings in strict mode"
  2. Modified pkg/cli/compile_command.go:

    • Updated both calls to runZizmor() to pass the strict flag
  3. Modified pkg/cli/zizmor_test.go:

    • Updated tests to handle the new return signature (warning count)
    • Added validation for non-negative warning counts
  4. Added pkg/workflow/strict_mode_zizmor_test.go:

    • New test file documenting expected behavior of strict mode with zizmor
  5. Modified pkg/workflow/strict_mode.go:

    • Added documentation comment explaining that strict mode affects zizmor behavior

Behavior Changes

Scenario Before After
Non-strict mode + zizmor findings ✅ Success (findings reported) ✅ Success (findings reported) - No change
Strict mode + zizmor findings ✅ Success (findings reported) Compilation fails - New behavior
Strict mode without zizmor ✅ Validates permissions/network only ✅ Validates permissions/network only - No change

Usage Example

# Enforce zero security findings
gh aw compile --strict --zizmor

# Report findings without failing
gh aw compile --zizmor

# Strict mode without zizmor
gh aw compile --strict
Original prompt

Update "strict" mode in compiler such that zizmor warnings/errors are not allowed and are treated S errors (if zizmor is enabled)

Run recompile and report current warnings status.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 3 commits October 28, 2025 01:10
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Update strict mode in compiler for zizmor handling Add strict mode enforcement for zizmor security findings Oct 28, 2025
Copilot AI requested a review from pelikhan October 28, 2025 01:19
@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot merge main, recompile

Copilot AI and others added 2 commits October 28, 2025 01:53
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Oct 28, 2025

@copilot merge main, recompile

Merged main and recompiled successfully in commit cd0b6be.

Fixed test workflows that were using github.actor and github.workflow (removed from allowed expressions in main):

  • test-all.md
  • test-template-github-actions-syntax.md
  • test-template-issue-context.md
  • test-template-pr-context.md

All workflows now compile successfully:

  • Main workflows: 56/56 ✅
  • Test workflows: 65/65 ✅

@pelikhan pelikhan marked this pull request as ready for review October 28, 2025 03:28
Copilot AI review requested due to automatic review settings October 28, 2025 03:28
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds strict mode enforcement for zizmor security findings, ensuring that when both --strict and --zizmor flags are enabled, any security warnings or errors detected by zizmor will cause compilation to fail rather than just being reported.

Key changes:

  • Modified zizmor integration to accept a strict parameter and treat findings as compilation errors when strict mode is enabled
  • Updated return signature of parseAndDisplayZizmorOutput to return warning count for proper error reporting
  • Added test coverage documenting the expected strict mode behavior with zizmor

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.

Show a summary per file
File Description
pkg/cli/zizmor.go Added strict mode parameter to runZizmor() and modified error handling to fail compilation when findings exist in strict mode
pkg/cli/compile_command.go Updated runZizmor() calls to pass the strict flag
pkg/cli/zizmor_test.go Updated tests to handle new return signature with warning count validation
pkg/workflow/strict_mode.go Added documentation explaining zizmor behavior in strict mode
pkg/workflow/strict_mode_zizmor_test.go Added test documenting expected strict mode + zizmor behavior
pkg/cli/workflows/*.md Removed deprecated github.actor and github.workflow expression references
docs/src/content/docs/reference/frontmatter-full.md Simplified toolsets documentation

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions
Copy link
Copy Markdown
Contributor

Agentic Changeset Generator triggered by this pull request.

@pelikhan pelikhan merged commit 7ae0eaa into main Oct 28, 2025
7 checks passed
@pelikhan pelikhan deleted the copilot/update-compiler-strict-mode branch October 28, 2025 03:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants