Fix markdownlint CI workflow (startup_failure since inception)

[danmoseley]

Problem

The markdownlint CI workflow has never worked — all 117 runs since it was added in PR #534 (April 17) show startup_failure. The root cause is that it uses third-party actions (tj-actions/changed-files and DavidAnson/markdownlint-cli2-action) that are not in the dotnet org's allowed actions list.

Fix

Workflow changes

Replace third-party actions with inline steps using only first-party actions already in the repo's allowlist:

• actions/checkout + actions/setup-node (both already approved, SHA-pinned)
• git diff for changed file detection (replaces tj-actions/changed-files)
• npx markdownlint-cli2 for linting (replaces DavidAnson/markdownlint-cli2-action)
• When config or workflow files change, lint all .md files (not just changed ones)

This follows the same pattern used by dotnet/runtime's working markdownlint workflow.

Config changes

Suppress rules in .markdownlint-cli2.jsonc that conflict with established repo conventions:

• MD001 (heading increment), MD007 (list indentation), MD014 (dollar signs in commands)
• MD022 (blanks around headings), MD026 (trailing punctuation in headings)
• MD028 (blank lines in blockquotes), MD029 (ordered list prefix style)
• MD031 (blanks around fences), MD032 (blanks around lists)
• MD034 (bare URLs), MD036 (emphasis as heading)
• MD040 (code fence language), MD056 (table column count)
• MD058 (blanks around tables), MD060 (table column style)

Mechanical fixes across 9 files

• MD009: removed trailing whitespace (4 files)
• MD012: reduced consecutive blank lines (3 files)
• MD038: fixed space in code span (1 file)
• MD047: added missing trailing newline (3 files)

Validation

• Ran markdownlint-cli2 locally against all .md files in the repo: 0 errors
• Both actions/checkout and actions/setup-node SHAs are already in the repo's GitHub Actions allowlist

[Copilot]

Pull request overview

This PR fixes the repository’s markdownlint CI workflow by removing disallowed third-party GitHub Actions and replacing them with inline git diff-based change detection and an npx markdownlint-cli2 invocation. It also updates the markdownlint-cli2 configuration to suppress several rules that conflict with established repo Markdown formatting conventions.

Changes:

• Replace tj-actions/changed-files and DavidAnson/markdownlint-cli2-action with git diff + npx markdownlint-cli2 in the CI workflow.
• Add actions/setup-node to support running markdownlint via Node tooling.
• Disable additional markdownlint rules in .markdownlint-cli2.jsonc to avoid large volumes of pre-existing violations.

Show a summary per file

File	Description
.markdownlint-cli2.jsonc	Disables additional markdownlint rules to align linting with existing repo Markdown conventions.
.github/workflows/markdownlint.yml	Reworks the workflow to avoid disallowed third-party actions by using git diff for change detection and npx markdownlint-cli2 for linting.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 2/2 changed files
• Comments generated: 3

[github-actions]

Skill Coverage Report

	Plugin	Skill	Covered	Coverage
⚠️	dotnet-ai	technology-selection	18/24	75%
⚠️	dotnet-diag	analyzing-dotnet-performance	12/21	57.1%
✅	dotnet-diag	microbenchmarking	3/3	100%
⚠️	dotnet-test	generate-testability-wrappers	14/22	63.6%
✅	dotnet-upgrade	migrate-dotnet10-to-dotnet11	5/6	83.3%

Uncovered: dotnet-ai/technology-selection

• [Validation] API keys are loaded from secure sources — not in source code or committed config files (line 314)
• [Validation] Non-deterministic outputs have validation and fallback paths (line 319)
• [Validation] dotnet build -c Release -warnaserror completes cleanly (line 320)
• [Pitfall] Over-engineering with LLMs (line 348)
• [Pitfall] Cold start latency on ML.NET models (line 354)
• [WorkflowStep] Step 3: Implement with guardrails (line 134)

Uncovered: dotnet-diag/analyzing-dotnet-performance

• [Validation] All critical patterns were checked (from reference files or inline recipes) (line 178)
• [Validation] Topic-specific recipes run only when matching signals detected (line 179)
• [Validation] Each finding includes a concrete code fix (line 180)
• [Validation] Scan execution checklist is complete (all recipes run) (line 181)
• [Validation] Summary table included at end (line 182)
• [Pitfall] Suggesting ConfigureAwait(false) in app code (line 191)
• [Pitfall] Recommending ValueTask everywhere (line 192)
• [Pitfall] Flagging new HttpClient() in DI services (line 193)
• [Pitfall] Suggesting CollectionsMarshal.AsSpan broadly (line 195)

Uncovered: dotnet-test/generate-testability-wrappers

• [Validation] DI registration uses AddSingleton for stateless wrappers, AddTransient for stateful ones (line 217)
• [Validation] Ambient context pattern includes AsyncLocal<T>, scoped disposal, and trade-off explanation (line 220)
• [Pitfall] Ambient context without AsyncLocal (line 231)
• [WorkflowStep] Step 5: Generate ambient context alternative (when DI is not available) (line 176)
• [CodePattern] sealed (line 120)
• [CodePattern] Assert.Equal (line 167)
• [CodePattern] Assert.True (line 81)
• [CodePattern] readonly (line 180)

Uncovered: dotnet-upgrade/migrate-dotnet10-to-dotnet11

• [WorkflowStep] Step 5: Update infrastructure (line 140)

[noahfalk]

The markdown changes looked fine. I didn't look at the yml CI portion.

[danmoseley]

/evaluate

[github-actions]

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
microbenchmarking	Investigate runtime upgrade performance impact	3.0/5 → 3.7/5 🟢	✅ microbenchmarking; tools: skill, read_bash / ✅ microbenchmarking; tools: read_bash, skill	✅ 0.12	❌
technology-selection	ML.NET classification on tabular data	3.0/5 ⏰ → 4.0/5 ⏰ 🟢	✅ technology-selection; tools: skill, stop_bash / ✅ technology-selection; tools: skill	🟡 0.42	✅ [1]
technology-selection	LLM integration with MEAI abstraction	1.0/5 → 1.0/5	⚠️ NOT ACTIVATED	🟡 0.42	❌ [2]
technology-selection	Reject LLM for tabular classification	3.0/5 → 3.7/5 🟢	✅ technology-selection; tools: skill, create / ✅ technology-selection; tools: skill	🟡 0.42	✅ [3]
technology-selection	Agentic workflow with guardrails	2.0/5 → 2.7/5 🟢	✅ technology-selection; tools: skill, create	🟡 0.42	✅ [4]
technology-selection	Natural-language scenario decomposition — RAG chatbot	3.3/5 → 4.3/5 🟢	✅ technology-selection; tools: skill, bash	🟡 0.42	✅ [5]
technology-selection	RAG pipeline with vector search	3.7/5 → 5.0/5 🟢	✅ technology-selection; tools: skill	🟡 0.42	✅ [6]
check-bin-obj-clash	Diagnose bin/obj output path clashes	3.3/5 → 5.0/5 🟢	✅ check-bin-obj-clash; tools: skill, glob / ✅ check-bin-obj-clash; tools: skill	✅ 0.12	✅ [7]
analyzing-dotnet-performance	Detects compiled regex startup budget and regex chain allocations	4.3/5 → 5.0/5 🟢	✅ analyzing-dotnet-performance; tools: skill, bash	✅ 0.13	✅ [8]
analyzing-dotnet-performance	Detects CurrentCulture comparer and compiled regex budget in inflection rules	4.3/5 → 5.0/5 🟢	✅ analyzing-dotnet-performance; tools: skill, bash	✅ 0.13	✅ [9]
analyzing-dotnet-performance	Finds per-call Dictionary allocation not hoisted to static	4.7/5 → 5.0/5 🟢	✅ analyzing-dotnet-performance; tools: skill, bash	✅ 0.13	❌ [10]
analyzing-dotnet-performance	Catches compound allocations in recursive number converter with ToLower	3.0/5 → 4.7/5 🟢	✅ analyzing-dotnet-performance; tools: skill, bash	✅ 0.13	✅
analyzing-dotnet-performance	Finds StringComparison.Ordinal missing and FrozenDictionary opportunities	5.0/5 → 5.0/5	✅ analyzing-dotnet-performance; tools: skill, bash	✅ 0.13	❌ [11]
analyzing-dotnet-performance	Detects Aggregate+Replace chain and struct missing IEquatable	4.0/5 → 4.3/5 🟢	✅ analyzing-dotnet-performance; tools: skill, bash	✅ 0.13	✅ [12]
analyzing-dotnet-performance	Finds branched Replace chain in format string manipulation	2.7/5 → 4.0/5 🟢	✅ analyzing-dotnet-performance; tools: skill, bash	✅ 0.13	✅ [13]
analyzing-dotnet-performance	Catches LINQ on hot-path string processing and All(char.IsUpper)	4.3/5 → 5.0/5 🟢	✅ analyzing-dotnet-performance; tools: skill, bash	✅ 0.13	✅ [14]
analyzing-dotnet-performance	Detects LINQ pipeline in TimeSpan formatting and collection processing	5.0/5 → 3.7/5 🔴	✅ analyzing-dotnet-performance; tools: skill, bash	✅ 0.13	❌
analyzing-dotnet-performance	Flags Span inconsistencies and compound method chains in truncation library	4.0/5 → 4.3/5 🟢	✅ analyzing-dotnet-performance; tools: skill, grep, bash / ✅ analyzing-dotnet-performance; tools: skill, bash, grep	✅ 0.13	❌ [15]
analyzing-dotnet-performance	Identifies unsealed leaf classes and locale hierarchy patterns	3.3/5 → 5.0/5 🟢	✅ analyzing-dotnet-performance; tools: skill, bash	✅ 0.13	✅
generate-testability-wrappers	Generate TimeProvider adoption for DateTime.UtcNow	3.0/5 ⏰ → 5.0/5 🟢	✅ generate-testability-wrappers; tools: skill / ✅ generate-testability-wrappers; tools: glob, skill, grep, edit	✅ 0.08	✅ [16]
generate-testability-wrappers	Generate custom Environment wrapper	3.0/5 → 5.0/5 🟢	✅ generate-testability-wrappers; tools: skill, glob, bash	✅ 0.08	✅
generate-testability-wrappers	Recommend System.IO.Abstractions for file system calls	3.3/5 → 5.0/5 🟢	✅ generate-testability-wrappers; tools: skill, report_intent, view, glob, bash, edit, grep / ✅ generate-testability-wrappers; tools: report_intent, skill	✅ 0.08	✅
generate-testability-wrappers	Decline wrapper generation for already-abstracted code	3.7/5 → 5.0/5 🟢	✅ generate-testability-wrappers; tools: skill	✅ 0.08	✅ [17]
migrate-dotnet10-to-dotnet11	Console app with compression and TAR operations	5.0/5 → 4.0/5 🔴	✅ migrate-dotnet10-to-dotnet11; tools: skill, view	✅ 0.03	❌
migrate-dotnet10-to-dotnet11	C# 15 compiler breaking changes — Span safe-context, nameof, with()	3.0/5 → 5.0/5 🟢	✅ migrate-dotnet10-to-dotnet11; tools: skill, report_intent, view / ✅ migrate-dotnet10-to-dotnet11; tools: report_intent, skill, view	✅ 0.03	✅
migrate-dotnet10-to-dotnet11	EF Core app with Cosmos DB provider using sync APIs	4.0/5 → 5.0/5 🟢	✅ migrate-dotnet10-to-dotnet11; tools: report_intent, skill, view	✅ 0.03	❌ [18]
migrate-dotnet10-to-dotnet11	Deployment to older hardware with minimum requirement changes	2.0/5 → 5.0/5 🟢	✅ migrate-dotnet10-to-dotnet11; tools: report_intent, skill, view	✅ 0.03	✅
migrate-dotnet10-to-dotnet11	Cryptography app using DSA on macOS	3.7/5 → 4.0/5 🟢	✅ migrate-dotnet10-to-dotnet11; tools: report_intent, skill, view	✅ 0.03	❌ [19]
migrate-dotnet10-to-dotnet11	Basic TFM update with Docker and global.json	4.7/5 → 5.0/5 🟢	✅ migrate-dotnet10-to-dotnet11; tools: skill / ✅ migrate-dotnet10-to-dotnet11; tools: skill, report_intent	✅ 0.03	❌ [20]
migrate-dotnet10-to-dotnet11	C# 15 dynamic operator and ref readonly delegate issues	2.7/5 → 5.0/5 🟢	✅ migrate-dotnet10-to-dotnet11; tools: report_intent, skill, view	✅ 0.03	✅

[1] ⚠️ High run-to-run variance (CV=0.86) — consider re-running with --runs 5
[2] ⚠️ High run-to-run variance (CV=1.34) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -0.9% due to: tokens (131183 → 145944)
[3] ⚠️ High run-to-run variance (CV=0.51) — consider re-running with --runs 5
[4] ⚠️ High run-to-run variance (CV=1.82) — consider re-running with --runs 5
[5] ⚠️ High run-to-run variance (CV=1.39) — consider re-running with --runs 5
[6] ⚠️ High run-to-run variance (CV=0.53) — consider re-running with --runs 5
[7] ⚠️ High run-to-run variance (CV=1.49) — consider re-running with --runs 5
[8] ⚠️ High run-to-run variance (CV=1.85) — consider re-running with --runs 5
[9] ⚠️ High run-to-run variance (CV=1.98) — consider re-running with --runs 5
[10] ⚠️ High run-to-run variance (CV=3.04) — consider re-running with --runs 5. (Plugin) Quality improved but weighted score is -4.2% due to: tokens (28919 → 89997), tool calls (2 → 5), time (29.6s → 65.7s)
[11] (Plugin) Quality unchanged but weighted score is -8.8% due to: tokens (29370 → 131780), tool calls (2 → 8), time (36.1s → 69.1s)
[12] ⚠️ High run-to-run variance (CV=0.69) — consider re-running with --runs 5
[13] ⚠️ High run-to-run variance (CV=0.57) — consider re-running with --runs 5
[14] ⚠️ High run-to-run variance (CV=1.87) — consider re-running with --runs 5
[15] ⚠️ High run-to-run variance (CV=5.08) — consider re-running with --runs 5. (Plugin) Quality unchanged but weighted score is -4.7% due to: tokens (43797 → 129774), tool calls (3 → 7), time (28.1s → 77.8s)
[16] ⚠️ High run-to-run variance (CV=0.54) — consider re-running with --runs 5
[17] ⚠️ High run-to-run variance (CV=1.53) — consider re-running with --runs 5
[18] (Isolated) Quality improved but weighted score is -12.2% due to: quality, tokens (13352 → 46402), tool calls (0 → 4)
[19] ⚠️ High run-to-run variance (CV=0.83) — consider re-running with --runs 5. (Isolated) Quality improved but weighted score is -10.8% due to: judgment, quality
[20] ⚠️ High run-to-run variance (CV=5.85) — consider re-running with --runs 5. (Plugin) Quality unchanged but weighted score is -6.0% due to: tokens (12262 → 31306), tool calls (0 → 1), time (7.8s → 17.7s)

⏰ timeout — run(s) hit the (240s, 360s, 600s) scenario timeout limit; scoring may be impacted by aborting model execution before it could produce its full output (increase via timeout in eval.yaml)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

▶ Sessions Visualisation -- interactive replay of all evaluation sessions

[danmoseley]

Oops, by resolving feedback before I pushed the fixes, this merged...