[aw-compat] Daily compat report (2026-05-13): 17/20 pass; playwright MCP→CLI codemod missing; strict:false gap in pull_request_target codemo [Content truncated due to length]

[github-actions]

Overview

Daily cross-repository compatibility run on 2026-05-13 built gh-aw at commit ef4ca4e and replayed gh aw compile --strict followed by gh aw fix --write + recompile against the top 20 public repositories (ranked by stars) whose .github/workflows/*.lock.yml files identify them as gh-aw consumers.

The good news: 17 of 20 (85%) compile cleanly today, with codemods auto-fixing 2 of those. 3 repositories still fail after gh aw fix --write, and they expose two concrete gaps worth fixing in this repo.

Summary

Outcome	Count	Notes
Pass on first compile	15	No changes needed
Fixed by gh aw fix --write	2	dotnet/runtime, storybookjs/storybook
Still failing after fix	3	Detailed below
Regressed after fix	0	–
Clone failed	0	–

gh-aw build version: gh aw version ef4ca4e

Actionable findings

1. Missing codemod: playwright MCP-mode → CLI-mode migration

• Symptom: strict mode: tools.playwright: MCP mode is deprecated. Migrate to CLI mode by adding 'mode: cli' to your playwright configuration.
• Observed in: rancher/dashboard (594 ⭐), file .github/workflows/daily-accessibility-review.md
• Why it matters: gh aw fix --write reports "No fixes needed" yet gh aw compile --strict rejects the same file. The deprecation message itself is concrete enough to be a codemod, and any other consumer using tools.playwright without mode: cli will hit the identical wall.
• Proposed codemod ID: playwright-mcp-to-cli-migration
  • Behavior: when tools.playwright is present and tools.playwright.mode is not cli, set mode: cli. The deprecation text already prescribes the exact mutation.
  • Optional follow-up: flag (warning, not error) bash steps that still invoke MCP browser tool names, since the prompt-side rewrite is harder to automate.

2. Codemod gap: pull-request-target-checkout-false refuses to run under strict: false, but --strict still hard-errors

• Symptom: pull_request_target trigger with checkout enabled is extremely insecure. raised as an error even after gh aw fix --write.
• Observed in: Azure/azure-sdk-for-js (2,293 ⭐), file .github/workflows/mgmt-review.md
• Root cause (verified by reading the codemod source):
  • pkg/cli/codemod_pull_request_target_checkout_false.go:22 early-returns whenever isFrontmatterStrictFalse(frontmatter) returns true.
  • mgmt-review.md sets strict: false in frontmatter.
  • But pkg/workflow/pull_request_target_validation.go decides to emit a hard error based on the compiler's c.strictMode field (driven by the --strict CLI flag / default), not the per-workflow strict: false setting. The two signals are not consistent.
• The codemod did succeed on 6 sibling workflows in the same repo (archie.md, dash.md, dexter.md, scribe.md, sentinel.md, tester.md) — mgmt-review.md is the only one with strict: false set, which is the discriminator.
• Two valid fixes (pick one):
  1. Make the codemod run unconditionally for pull_request_target checkout (drop the isFrontmatterStrictFalse gate in codemod_pull_request_target_checkout_false.go). The user can always revert; the inserted checkout: false is safe.
  2. Or, honour per-workflow strict: false inside validatePullRequestTargetTrigger so the compile path agrees with the codemod's notion of opt-out.
• Note: the same isFrontmatterStrictFalse early-exit also exists in codemod_sandbox_agent_false_removal.go and codemod_sandbox_mcp_internal.go — worth reviewing for the same hazard.

3. User-facing: import file not found (not a codemod problem)

• Symptom: error: import file not found on imports: - shared/tavily-mcp.md etc.
• Observed in: microsoft/wassette .github/workflows/scout.md
• Why it matters here: gh aw fix --write claimed it fixed all 8 wassette workflows, and 7 of them now pass. The scout.md failure is unrelated to codemods — the workflow author needs to vendor the shared imports or remove them. No gh-aw-side change needed, but flagging it because the "fixed 8 of 8" message was misleadingly cheerful.

What's working well

gh aw fix --write applied the following codemods successfully across the sample:

Codemod application counts

• Add checkout: false for pull_request_target — 7 applications
• Migrate timeout_minutes to timeout-minutes — 7 applications
• Move roles to on.roles — 5 applications
• Transform activation outputs to sanitized step — 4 applications
• Rename 'rate-limit' to 'user-rate-limit' — 3 applications
• Migrate on.command to on.slash_command — 2 applications
• Migrate tools.serena or engine.tools.serena to shared Serena import — 1 application
• Convert expires integer to day string — 1 application
• Convert write permissions to read — 1 application
• Add workflow_run branch restrictions — 1 application

This is what auto-upgraded dotnet/runtime and storybookjs/storybook from failing to passing.

Selected repositories

Top 20 by stars (selected from 202 candidates discovered via gh search code 'gh-aw-metadata filename:.lock.yml')

Repo	Stars	Outcome
f/prompts.chat	162,157	pass
storybookjs/storybook	89,895	fixed by codemod
dotnet/maui	23,246	pass
dotnet/runtime	17,880	fixed by codemod
prowler-cloud/prowler	13,784	pass
Azure/azure-sdk-for-net	5,985	pass
dotnet/sdk	3,125	pass
apache/cloudstack	2,899	pass
Azure/azure-sdk-for-js	2,293	still failing (finding #2)
dotnet/skills	1,686	pass
ionide/ionide-vscode-fsharp	893	pass
microsoft/wassette	886	still failing (finding #3)
rancher/dashboard	594	still failing (finding #1)
fsprojects/FsHttp	498	pass
ionide/FsAutoComplete	483	pass
fsprojects/FSharp.TypeProviders.SDK	304	pass
fsprojects/FSharpx.Collections	253	pass
benbalter/word-to-markdown-js	210	pass
microsoft/vscode-black-formatter	195	pass
MaggieAppleton/maggieappleton.com-V3	147	pass

Failure diagnostics — raw error lines

Azure/azure-sdk-for-js (mgmt-review.md)

.github/workflows/mgmt-review.md:1:1: error: pull_request_target trigger with checkout enabled is extremely insecure.

6 sibling workflows fixed automatically; this one alone slipped through because of strict: false in frontmatter.

microsoft/wassette (scout.md)

.github/workflows/scout.md:17:5: error: import file not found
  - shared/tavily-mcp.md

The other 7 wassette workflows compile after codemod fixes for timeout_minutes, roles, on.command, and activation-output rewrites.

rancher/dashboard (daily-accessibility-review.md)

strict mode: tools.playwright: MCP mode is deprecated. Migrate to CLI mode by adding `mode: cli` to your playwright configuration.

gh aw fix --write reports "No fixes needed".

Recommended next steps

1. Add playwright-mcp-to-cli-migration codemod — straightforward, deterministic mutation; deprecation message already specifies the exact fix.
2. Reconcile strict: false semantics between codemods and the validator — either codemods should run regardless, or strict-mode compile should respect the opt-out. Today they disagree and trap users.
3. Audit the other isFrontmatterStrictFalse usages (codemod_sandbox_agent_false_removal.go, codemod_sandbox_mcp_internal.go) for the same trap.

References

• Run artifacts: /tmp/gh-aw/cache-memory/aw-compat/runs/2026-05-13-09-33-31-433/
• Workflow run: §25790700506

Generated by Daily AW Cross-Repo Compile Check · ● 21.1M · ◷

• expires on May 20, 2026, 9:46 AM UTC

[pelikhan]

@copilot playwright MCP should be allowed in strict mode, not an error. No codemod needed.

[pelikhan]

@copilot honour per-workflow strict: false inside validatePullRequestTargetTrigger so the compile path agrees with the codemod's notion of opt-out.