Port CI analysis Copilot skill and default it to aspnetcore#65464
Conversation
Co-authored-by: wtgodbe <14283640+wtgodbe@users.noreply.github.com>
wtgodbe
left a comment
There was a problem hiding this comment.
Please make the requested changes to reference aspnetcore rather than runtime
Co-authored-by: wtgodbe <14283640+wtgodbe@users.noreply.github.com>
Updated in commit c7f2785. I applied the requested aspnetcore-focused reference changes across the reviewed skill docs. |
There was a problem hiding this comment.
Pull request overview
Ports the ci-analysis Copilot skill into aspnetcore and updates defaults/documentation so CI triage targets dotnet/aspnetcore out of the box.
Changes:
- Added the
Get-CIStatus.ps1CI triage script (AzDO + Helix + optional known-issue correlation) with aspnetcore-first defaults. - Added skill documentation (
SKILL.md) describing usage patterns and recommendations. - Added reference guides covering investigation workflows (SQL tracking, manual investigation, Helix artifacts, Azure CLI, build progression, etc.).
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| .github/skills/ci-analysis/scripts/Get-CIStatus.ps1 | Adds the CI analysis script and sets default repository/org/project behavior. |
| .github/skills/ci-analysis/SKILL.md | Documents how to use the skill/script and interpret outputs. |
| .github/skills/ci-analysis/references/azdo-helix-reference.md | Reference info for AzDO/Helix concepts and aspnetcore pipeline IDs. |
| .github/skills/ci-analysis/references/azure-cli.md | Azure CLI fallback guidance for deeper AzDO investigation. |
| .github/skills/ci-analysis/references/binlog-comparison.md | Workflow for comparing MSBuild binlogs between passing/failing runs. |
| .github/skills/ci-analysis/references/build-progression-analysis.md | Workflow for correlating failures across a PR’s build history. |
| .github/skills/ci-analysis/references/delegation-patterns.md | Patterns for delegating mechanical CI data gathering to subagents. |
| .github/skills/ci-analysis/references/helix-artifacts.md | Guide to locating and interpreting Helix work item artifacts (binlogs, results, dumps). |
| .github/skills/ci-analysis/references/manual-investigation.md | Manual fallback steps when script output is insufficient. |
| .github/skills/ci-analysis/references/sql-tracking.md | SQL schema and queries for tracking failures/known-issue mapping during investigations. |
| param( | ||
| [string]$TestName = "", | ||
| [string]$ErrorMessage = "", | ||
| [string]$Repository = $script:Repository, |
There was a problem hiding this comment.
Show-KnownIssues defaults -Repository to $script:Repository, but this script never sets $script:Repository. That means calls like Show-KnownIssues ... end up passing $null into Search-KnownIssues, and gh issue list --repo $Repository will fail. Default this parameter to the script’s param() $Repository value (or a literal default), or assign $script:Repository = $Repository early and use that consistently.
| [string]$Repository = $script:Repository, | |
| [string]$Repository = $env:GITHUB_REPOSITORY, |
| [int]$MaxJobs = 5, | ||
| [int]$MaxFailureLines = 50, | ||
| [int]$TimeoutSec = 30, | ||
| [int]$ContextLines = 0, | ||
| [switch]$NoCache, |
There was a problem hiding this comment.
-ContextLines is documented and exposed as a top-level parameter but isn’t used anywhere, so changing it has no effect. Wire this through to Extract-BuildErrors (and any other log extraction) so the output matches the parameter contract (including the documented default of 0).
| function Extract-BuildErrors { | ||
| param( | ||
| [string]$LogContent, | ||
| [int]$Context = 5 |
There was a problem hiding this comment.
Extract-BuildErrors defaults Context to 5, which conflicts with the script’s -ContextLines default of 0 and (currently) causes context lines to be included even when the user didn’t request them. Consider defaulting Context to 0 and passing the top-level -ContextLines value explicitly when calling this helper.
| [int]$Context = 5 | |
| [int]$Context = 0 |
|
|
||
| ## Examining Pipeline YAML | ||
|
|
||
| All dotnet repos that use arcade put their pipeline definitions under `eng/pipelines/`. Use `az pipelines show` to find the YAML file path, then fetch it: |
There was a problem hiding this comment.
The statement that all dotnet repos using Arcade keep pipeline YAML under eng/pipelines/ isn’t accurate for this repo (aspnetcore uses .azure/pipelines/, and eng/pipelines/ doesn’t exist here). Please soften/reword this to avoid sending users to a non-existent directory, while still mentioning eng/pipelines/ as common in other repos/VMR.
| All dotnet repos that use arcade put their pipeline definitions under `eng/pipelines/`. Use `az pipelines show` to find the YAML file path, then fetch it: | |
| In many dotnet repos that use Arcade (and in the VMR), pipeline definitions are kept under `eng/pipelines/`, but individual repos may use different locations (for example, this repo uses `.azure/pipelines/`). Use `az pipelines show` to find the YAML file path, then fetch it: |
| | `-SearchMihuBot` | Search MihuBot for related issues | | ||
|
|
There was a problem hiding this comment.
-SearchMihuBot triggers calls to a third-party endpoint (mihubot.xyz) with derived search terms/context from failures. The SKILL docs currently don’t mention the external data transfer; please add an explicit note/warning here so users understand what’s being sent and can make an informed choice (and consider suggesting it only for public repos/logs).
| | `-SearchMihuBot` | Search MihuBot for related issues | | |
| | `-SearchMihuBot` | Search MihuBot for related issues (see note below about external data transfer) | | |
| > ⚠️ **Privacy note for `-SearchMihuBot`:** Enabling this option sends derived search terms and brief context about CI failures (e.g., test names, error messages, and basic repository/PR identifiers) to a third-party service at `mihubot.xyz` to look up related issues. Avoid using this flag with private or sensitive repositories/logs unless this data sharing is acceptable under your security and privacy requirements; it is generally safest to use it only with public repos and non-sensitive CI output. |
Bring runtime ci-analysis skill to aspnetcore defaults
Description
This ports the
ci-analysisCopilot skill fromdotnet/runtimeinto this repo and adjusts the default experience to targetdotnet/aspnetcoreout of the box.Skill import
.github/skills/ci-analysis/with:SKILL.mdscripts/Get-CIStatus.ps1references/*docs used by the skillAspNetCore-first defaults (explicit modifications)
dotnet/runtimetodotnet/aspnetcorein:SKILL.mdquick-start + parameter tablescripts/Get-CIStatus.ps1parameter defaults/help text/examplesreferences/sql-tracking.mdSQL default repo exampleReview-feedback updates for aspnetcore-specific guidance
ActiveIssueguidance withSkipOnHelixandQuarantinedTestinSKILL.mdreferences/azdo-helix-reference.mdto aspnetcore pipeline IDs/names:83-aspnetcore-ci86-aspnetcore-quarantined-pr87-aspnetcore-components-e2e318-aspnetcore-template-tests-prhttps://github.com/dotnet/aspnetcore/blob/main/docs/area-owners.mdhttps://github.com/dotnet/aspnetcore/blob/main/.azure/pipelines/ci-public.ymlBehavioral impact
-Repositorynow analyzes aspnetcore PRs/builds by default; runtime/sdk/roslyn remain supported via explicit-Repository.💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.