Skip to content

ci: scope system-test runs to only affected code paths#2942

Merged
ksail-bot[bot] merged 4 commits intomainfrom
copilot/update-ci-workflow-system-tests
Mar 10, 2026
Merged

ci: scope system-test runs to only affected code paths#2942
ksail-bot[bot] merged 4 commits intomainfrom
copilot/update-ci-workflow-system-tests

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 9, 2026

System tests run the full matrix (4 distributions × 5 configs × 2 init modes + Hetzner variants) on every non-test Go source change, including code paths they don't exercise.

Adds a system-test path filter that excludes packages verified to be import-isolated from the cluster/workload CLI commands under test:

  • Excluded packages: pkg/cli/cmd/chat/**, pkg/cli/ui/chat/**, pkg/svc/chat/**, pkg/svc/mcp/**, pkg/toolgen/**, pkg/cli/cmd/cipher/**, docs/*.go, schemas/**/*.go
  • Added triggers: .github/fixtures/** (test fixtures were missing from the original source filter), .github/workflows/ci.yaml (workflow changes should trigger system tests to catch broken logic)
  • Updated jobs: warm-helm-cache, warm-mirror-cache, system-test now gate on system-test instead of source
  • Removed: the now-unused source path filter output and definition (no remaining consumers)

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Add a `system-test` path filter that excludes code not exercised by
system tests (chat, MCP, cipher, toolgen, doc/schema generators).
Include `.github/fixtures/**` since test fixtures should trigger runs.
Update warm-helm-cache, warm-mirror-cache, and system-test jobs to use
the new filter instead of the broader `source` filter.

Co-authored-by: devantler <26203420+devantler@users.noreply.github.com>
Copilot AI changed the title [WIP] Update CI workflow to optimize system-tests execution ci: scope system-test runs to only affected code paths Mar 9, 2026
@devantler
Copy link
Copy Markdown
Contributor

@copilot if the source filter is no longer in use it should not be kept. Investigate if we can do a little cleanup, or if I am overlooking something. If cleanuo is possible go do it

The `source` filter and output were superseded by `system-test` and
are no longer referenced by any job. Remove both the output declaration
and the filter definition.

Co-authored-by: devantler <26203420+devantler@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 10, 2026

@copilot if the source filter is no longer in use it should not be kept. Investigate if we can do a little cleanup, or if I am overlooking something. If cleanuo is possible go do it

You're right — source had no remaining consumers after the switch to system-test. Removed both the output declaration and filter definition in 7de3bde.

@devantler devantler marked this pull request as ready for review March 10, 2026 00:04
@devantler devantler self-requested a review as a code owner March 10, 2026 00:04
Copilot AI review requested due to automatic review settings March 10, 2026 00:04
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 10, 2026

⚠️MegaLinter analysis: Success with warnings

⚠️ MARKDOWN / markdownlint - 60 errors
.agents/skills/astro/SKILL.md:64 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/astro/SKILL.md:70 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/astro/SKILL.md:76 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/astro/SKILL.md:82 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/copilot-sdk/SKILL.md:879 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/find-skills/SKILL.md:60 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/find-skills/SKILL.md:77 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/find-skills/SKILL.md:127 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/gh-cli/SKILL.md:70 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/git-commit/SKILL.md:16 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/git-commit/SKILL.md:42 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/github-issues/SKILL.md:33 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/github-issues/SKILL.md:42 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/github-issues/SKILL.md:72 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/golang-pro/references/project-structure.md:5 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/golang-pro/references/project-structure.md:177 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/refactor/SKILL.md:37 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/refactor/SKILL.md:563 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.agents/skills/web-design-guidelines/SKILL.md:25 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/create-agentic-workflow.md:84 error MD036/no-emphasis-as-heading Emphasis used instead of a heading [Context: "Read the gh-aw instructions"]
.github/aw/create-shared-agentic-workflow.md:16 error MD036/no-emphasis-as-heading Emphasis used instead of a heading [Context: "Build on agentic workflows"]
.github/aw/create-shared-agentic-workflow.md:23 error MD036/no-emphasis-as-heading Emphasis used instead of a heading [Context: "Prefer Docker Solutions"]
.github/aw/create-shared-agentic-workflow.md:30 error MD036/no-emphasis-as-heading Emphasis used instead of a heading [Context: "Support Read-Only Tools"]
.github/aw/create-shared-agentic-workflow.md:37 error MD036/no-emphasis-as-heading Emphasis used instead of a heading [Context: "Move Write Operations to Safe ..."]
.github/aw/create-shared-agentic-workflow.md:44 error MD036/no-emphasis-as-heading Emphasis used instead of a heading [Context: "Process Agent Output in Safe J..."]
.github/aw/create-shared-agentic-workflow.md:57 error MD036/no-emphasis-as-heading Emphasis used instead of a heading [Context: "Documentation"]
.github/aw/create-shared-agentic-workflow.md:153 error MD036/no-emphasis-as-heading Emphasis used instead of a heading [Context: "Example: Processing Agent Outp..."]
.github/aw/debug-agentic-workflow.md:19 error MD036/no-emphasis-as-heading Emphasis used instead of a heading [Context: "Example: Debugging from a work..."]
.github/aw/debug-agentic-workflow.md:25 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/debug-agentic-workflow.md:39 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/debug-agentic-workflow.md:53 error MD036/no-emphasis-as-heading Emphasis used instead of a heading [Context: "Prerequisites"]
.github/aw/debug-agentic-workflow.md:60 error MD036/no-emphasis-as-heading Emphasis used instead of a heading [Context: "Key Commands Available"]
.github/aw/debug-agentic-workflow.md:91 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/debug-agentic-workflow.md:124 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/debug-agentic-workflow.md:162 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/debug-agentic-workflow.md:199 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/debug-agentic-workflow.md:208 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/debug-agentic-workflow.md:218 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/debug-agentic-workflow.md:256 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/debug-agentic-workflow.md:468 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/serena-tool.md:118 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/serena-tool.md:126 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/serena-tool.md:135 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/serena-tool.md:174 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/update-agentic-workflow.md:58 error MD036/no-emphasis-as-heading Emphasis used instead of a heading [Context: "Read the gh-aw instructions"]
.github/aw/update-agentic-workflow.md:523 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/aw/upgrade-agentic-workflows.md:13 error MD036/no-emphasis-as-heading Emphasis used instead of a heading [Context: "Prerequisites"]
.github/aw/upgrade-agentic-workflows.md:20 error MD036/no-emphasis-as-heading Emphasis used instead of a heading [Context: "Key Commands Available"]
.github/aw/upgrade-agentic-workflows.md:265 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/workflows/daily-workflow-maintenance.md:255 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
.github/workflows/weekly-promote-ksail.md:117 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "````"]
pkg/cli/cmd/cipher/BENCHMARKS.md:93 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
pkg/cli/cmd/cipher/BENCHMARKS.md:111 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
pkg/cli/cmd/cipher/BENCHMARKS.md:172 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
pkg/client/argocd/BENCHMARKS.md:148 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "````"]
pkg/client/docker/BENCHMARKS.md:61 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
pkg/client/helm/BENCHMARKS.md:60 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
pkg/client/kustomize/BENCHMARKS.md:51 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
pkg/fsutil/marshaller/BENCHMARKS.md:65 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
vsce/README.md:128 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]

✅ Linters with no issues

actionlint, git_diff, hadolint, jscpd, jsonlint, lychee, markdown-table-formatter, prettier, prettier, stylelint, syft, trivy-sbom, trufflehog, v8r, v8r, yamllint

See detailed reports in MegaLinter artifacts

MegaLinter is graciously provided by OX Security
Show us your support by starring ⭐ the repository

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the main CI workflow’s path filtering to better scope when merge-queue system-test-related jobs (cache warming + system tests) run.

Changes:

  • Replaces the source change-detection output with a new system-test output.
  • Adds a dedicated system-test paths-filter with explicit include/exclude patterns.
  • Updates warm-cache and system-test job conditions to key off needs.changes.outputs.system-test.

Comment thread .github/workflows/ci.yaml
@devantler
Copy link
Copy Markdown
Contributor

@copilot apply changes based on the comments in this thread

Changes to the workflow file itself should trigger system test runs
to catch broken workflow logic before merge.

Co-authored-by: devantler <26203420+devantler@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 10, 2026

@copilot apply changes based on the comments in this thread

Added .github/workflows/ci.yaml to the system-test filter in 136678c so workflow changes trigger system test runs and don't slip through unvalidated.

@ksail-bot ksail-bot Bot enabled auto-merge March 10, 2026 00:19
@github-project-automation github-project-automation Bot moved this from 🫴 Ready to 🚀 In Finalization in 🌊 Project Board Mar 10, 2026
@ksail-bot ksail-bot Bot added this pull request to the merge queue Mar 10, 2026
Merged via the queue into main with commit 0d9c4e9 Mar 10, 2026
24 checks passed
@ksail-bot ksail-bot Bot deleted the copilot/update-ci-workflow-system-tests branch March 10, 2026 02:40
@github-project-automation github-project-automation Bot moved this from 🚀 In Finalization to ✅ Done in 🌊 Project Board Mar 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

3 participants