Skip to content

Add daily-safe-output-integrator agentic workflow#21415

Merged
pelikhan merged 2 commits intomainfrom
copilot/create-safe-output-integrator
Mar 17, 2026
Merged

Add daily-safe-output-integrator agentic workflow#21415
pelikhan merged 2 commits intomainfrom
copilot/create-safe-output-integrator

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 17, 2026

Adds a daily agentic workflow that maintains test coverage parity between the compiler's supported safe-output types and the test fixtures in pkg/cli/workflows/.

What it does

  • Extracts all safe-output YAML keys from SafeOutputsConfig in pkg/workflow/compiler_types.go as the authoritative type list
  • Cross-references against existing test workflows in pkg/cli/workflows/ and Go test cases in compiler_safe_outputs_config_test.go
  • For each uncovered type: creates a test-copilot-<type>.md fixture and adds a TestAddHandlerManagerConfigEnvVar test case
  • Opens a PR with all new files, or calls noop if coverage is already complete

Files

File Purpose
daily-safe-output-integrator.md Workflow source with detection logic, per-type templates, and Go test patterns
daily-safe-output-integrator.lock.yml Compiled GitHub Actions workflow

Coverage gap (current)

13 safe-output types currently lack test workflows, including create-discussion, close-issue, update-pull-request, remove-labels, missing-data, and project management types (create-project, update-project, create-project-status-update). The workflow includes ready-to-use templates for each.

Original prompt

Here we are going to create a daily agentic workflow called Safe Output Integrator.

The purpose of this workflow is:

  • On each run, inspect the list of test workflows under pkg4/cli for xlac workflows.
  • Read the safe-output schema from main.json and schema.json.
  • Cross-correlate which safe-output values are already covered by existing tests.
  • Detect any safe-output values that are not covered.
  • For each missing safe-output:
    • Create a corresponding test workflow in the pkg/cli/workflow folder.
    • Add a test that verifies the compiler compiles that safe-output correctly according to the specification.
    • Ensure this also applies to all available features that are active in that safe-output.

Create a pull-request if there are any changes. Otherwise, call NOOP and notify that all tests are up to date.


📱 Kick off Copilot coding agent tasks wherever you are with GitHub Mobile, available on iOS and Android.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add safe output integrator workflow Add daily-safe-output-integrator agentic workflow Mar 17, 2026
Copilot AI requested a review from pelikhan March 17, 2026 16:32
@pelikhan pelikhan marked this pull request as ready for review March 17, 2026 16:43
Copilot AI review requested due to automatic review settings March 17, 2026 16:43
@pelikhan pelikhan merged commit 5798209 into main Mar 17, 2026
98 checks passed
@pelikhan pelikhan deleted the copilot/create-safe-output-integrator branch March 17, 2026 16:43
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new daily agentic workflow (“Daily Safe Output Integrator”) intended to keep safe-output test fixtures and compiler test coverage aligned with the compiler’s supported safe-output types.

Changes:

  • Adds the workflow source prompt/instructions for detecting missing safe-output coverage and generating fixtures/tests.
  • Adds the compiled GitHub Actions workflow lockfile generated from the source workflow.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
.github/workflows/daily-safe-output-integrator.md Defines the agentic workflow prompt, tool allowlist, and templates for generating missing safe-output fixtures and Go tests.
.github/workflows/daily-safe-output-integrator.lock.yml Generated compiled workflow that runs the agent daily and processes safe-output results.
Comments suppressed due to low confidence (1)

.github/workflows/daily-safe-output-integrator.md:83

  • The “Skip internal/special fields” guidance lists only a subset of non-output config keys, but SafeOutputsConfig contains many more configuration-only YAML keys (e.g. max-patch-size, runs-on, messages, mentions, footer, etc.). If the workflow truly relies on parsing YAML tags from SafeOutputsConfig, the skip logic should be complete (or expressed as an allowlist of pointer-to-config output types) to avoid generating fixtures/tests for config-only fields.
The keys you're looking for are the YAML field names in the `SafeOutputsConfig` struct (lines ~443-490). These are the authoritative list of supported safe-output types.

**Extract and record**: Every `yaml:"<key>,omitempty"` found in `SafeOutputsConfig`. Skip internal/special fields like `jobs`, `github-app`, `env`, `github-token`, `allowed-domains`, `allowed-github-references`, `staged`, and `threat-detection` — those are configuration, not output types.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

- "cat pkg/cli/workflows/*.md"
- "git status"
- "git diff --name-only"
- "python3 *"
- `submit-pull-request-review`
- `reply-to-pull-request-review-comment`
- `resolve-pull-request-review-thread`
- `create-code-scanning-alerts`
Comment on lines +84 to +103
The **testable safe-output types** are:
- `create-issue`
- `create-discussion`
- `update-discussion`
- `close-discussion`
- `close-issue`
- `close-pull-request`
- `mark-pull-request-as-ready-for-review`
- `add-comment`
- `create-pull-request`
- `create-pull-request-review-comment`
- `submit-pull-request-review`
- `reply-to-pull-request-review-comment`
- `resolve-pull-request-review-thread`
- `create-code-scanning-alerts`
- `autofix-code-scanning-alert`
- `add-labels`
- `remove-labels`
- `add-reviewer`
- `assign-milestone`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants