Fix wasm/native compiler divergence, add smoke workflow golden tests by Mossaka · Pull Request #16705 · github/gh-aw

Mossaka · 2026-02-19T01:33:23Z

Summary

Fix 3 bugs causing wasm compiler output to diverge from native compiler
Add smoke workflows (claude, codex, copilot) as golden test fixtures alongside the 16 synthetic fixtures from Add wasm compiler integration tests with golden file comparison #16643
Keep fixtures self-contained — production workflow changes don't break the build

Bug fixes (wasm compiler parity)

Fix pre_activation job divergence (`compiler_jobs.go`)

buildJobs() read frontmatter from disk via os.ReadFile(markdownPath) to decide whether to generate a pre_activation job. In wasm/string API mode, markdownPath is virtual — os.ReadFile fails, frontmatter is nil, and pre_activation is always generated. Fix: check c.contentOverride first, fall back to disk.

Fix contentOverride lifetime (`compiler_string_api.go`)

ParseWorkflowString cleared contentOverride via defer before CompileToYAML could use it. Moved cleanup to CompileToYAML so content remains available throughout compilation.

Fix security scanning in wasm (`compiler_orchestrator_engine.go`)

Security scanning of imported files used os.ReadFile directly, bypassing the virtual filesystem. Changed to parser.ReadFile which routes through readFileFunc (overridden in wasm builds).

Add parser.ReadFile wrapper (`virtual_fs.go`)

Exported ReadFile function routing through readFileFunc for both native and wasm builds.

Add WorkflowIdentifier to wasm binary (`cmd/gh-aw-wasm/main.go`)

Derive workflow identifier from filename for fuzzy cron schedule scattering.

Test fixtures (20 total)

Category	Count	Purpose
Synthetic fixtures	16	Engine types, triggers, features
Smoke workflows	4	Real-world coverage (claude, codex, copilot, test-tools)
Shared components	9	Import deps for smoke workflows

Fixtures are self-contained in testdata/ — editing .github/workflows/*.md will NOT break these tests.

Test plan

Go golden tests: 20 passed, 0 failed
Node.js wasm binary test: 20 passed, 0 failed, 0 skipped
Full unit test suite: 18/18 packages pass
CI: build-wasm, test, lint-go, lint-js all pass

🤖 Generated with Claude Code

Add all 130 production agentic workflows from .github/workflows/ as wasm golden test fixtures, alongside their 58 shared component dependencies. This increases coverage from 16 synthetic fixtures to 142 real-world production workflows (plus 5 that are gracefully skipped due to path security restrictions or missing external files). Key changes: - Copy all production .md workflows as test fixtures - Copy all shared/ components (37 root + 21 MCP configs) for import resolution - Fix fuzzy cron schedule scattering by passing WithWorkflowIdentifier() to the compiler (derived from fixture filename) - Gracefully skip workflows that can't compile via string API (path security, missing agent files) instead of hard-failing Coverage by engine: 72 copilot, 31 claude, 9 codex workflows Coverage by feature: schedule triggers, safe-outputs, imports, tools, network config, slash commands, label triggers, MCP servers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

This PR expands the wasm golden test coverage from 16 synthetic fixtures to 142 real production workflows, adds 58 shared components, and fixes a bug in fuzzy cron schedule compilation.

Changes:

Added 204 production workflow files (130 agentic workflows + 58 shared components + 16 original fixtures) as test inputs
Fixed fuzzy cron schedule compilation by passing workflow identifier to the compiler
Updated test to gracefully skip 5 workflows with path security restrictions instead of failing

Reviewed changes

Copilot reviewed 24 out of 313 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
codex-github-remote-mcp-test.golden	New golden output for Codex engine workflow testing GitHub remote MCP integration
code-scanning-fixer.golden	New golden output for automated code scanning alert fixing workflow
cli-consistency-checker.golden	New golden output for CLI consistency verification workflow
chroma-issue-indexer.golden	New golden output for issue indexing into Chroma vector database
changeset.golden	New golden output for automated changeset file generation
brave.golden	New golden output for Brave search agent workflow
artifacts-summary.golden	New golden output for artifacts usage reporting workflow
archie.golden	New golden output for Mermaid diagram generation agent
ai-moderator.golden	New golden output for AI-powered content moderation workflow

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…rkflows ## Bug fixes (wasm compiler parity) ### Fix pre_activation job divergence (compiler_jobs.go) buildJobs() read frontmatter from disk via os.ReadFile(markdownPath) to determine if pre_activation job should be generated. In wasm/string API mode, markdownPath is virtual — os.ReadFile fails, frontmatter is nil, and pre_activation is always generated even when the workflow uses safe-events-only triggers. Fix: check c.contentOverride first (set by ParseWorkflowString), falling back to disk read. ### Fix contentOverride lifetime (compiler_string_api.go) ParseWorkflowString cleared contentOverride via defer before CompileToYAML could use it. Moved the defer cleanup to CompileToYAML so the content remains available throughout the compilation pipeline. ### Fix security scanning in wasm (compiler_orchestrator_engine.go) Security scanning of imported files used os.ReadFile directly, bypassing the virtual filesystem. Changed to parser.ReadFile which routes through readFileFunc (overridden in wasm builds to check virtual files first). ### Add parser.ReadFile wrapper (virtual_fs.go) Exported ReadFile function that routes through readFileFunc, enabling both native (os.ReadFile) and wasm (virtual filesystem) builds to use the same API for file reading. ### Add WorkflowIdentifier to wasm binary (cmd/gh-aw-wasm/main.go) Derive workflow identifier from filename for fuzzy cron schedule scattering. Without this, workflows with schedule triggers fail to compile in the wasm binary. ## Test expansion ### Node.js wasm test improvements (scripts/test-wasm-golden.mjs) - Load ALL shared components as virtual files (not just imports) - Skip fixtures without golden files before attempting compilation - Simplified error handling: golden file presence determines skip vs fail Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

pelikhan

can we reduce the number of files. we don't need the entire zoo. a single workflow, or smoke workflow will suffice. we are not testing the wasm compiler.
these tests should not break on mondane changes to the .github/workflows/*.md . if they break the build everytime, it will create too much friction. best to keep a local "gold suite"
is there a mechnical way of fixing these test, not agentic.

pelikhan

see comments

Per review feedback: remove all 130 production workflows to avoid golden tests breaking on mundane .github/workflows/*.md changes. Keep a small, stable "gold suite" of: - 16 synthetic fixtures (engine types, triggers, features) - 4 smoke workflows (smoke-claude, smoke-codex, smoke-copilot, smoke-test-tools) for real-world coverage - 9 shared components needed by smoke workflows (from 58) This prevents the golden tests from creating friction when production workflows are edited — the fixtures are self-contained and only change when the compiler itself changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Documents directory structure, how to regenerate golden files, add new fixtures, fix regressions, and debug wasm-specific divergence. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

pelikhan

Please add a README.md, scripts to explain how to maintain, debug, refresh this test suite.

Copilot AI review requested due to automatic review settings February 19, 2026 01:33

Copilot AI reviewed Feb 19, 2026

View reviewed changes

Mossaka force-pushed the feat/wasm-golden-expand branch from 6df35c6 to 2c60b79 Compare February 19, 2026 02:04

pelikhan self-requested a review February 19, 2026 02:25

pelikhan reviewed Feb 19, 2026

View reviewed changes

pelikhan requested changes Feb 19, 2026

View reviewed changes

Mossaka and others added 3 commits February 19, 2026 03:50

Merge branch 'main' into feat/wasm-golden-expand

b4b3bf0

Merge branch 'main' into feat/wasm-golden-expand

02c5fbe

Mossaka changed the title ~~Expand wasm golden tests to 142 production workflows~~ Fix wasm/native compiler divergence, add smoke workflow golden tests Feb 19, 2026

Add README for wasm golden test suite maintenance

b09c8a2

Documents directory structure, how to regenerate golden files, add new fixtures, fix regressions, and debug wasm-specific divergence. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

pelikhan approved these changes Feb 19, 2026

View reviewed changes

Mossaka merged commit 2f5ad89 into main Feb 19, 2026
56 checks passed

Mossaka deleted the feat/wasm-golden-expand branch February 19, 2026 04:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix wasm/native compiler divergence, add smoke workflow golden tests#16705

Fix wasm/native compiler divergence, add smoke workflow golden tests#16705
Mossaka merged 6 commits intomainfrom
feat/wasm-golden-expand

Mossaka commented Feb 19, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

pelikhan left a comment •

edited

Loading

Uh oh!

pelikhan left a comment

Uh oh!

pelikhan left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

Conversation

Mossaka commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Bug fixes (wasm compiler parity)

Fix pre_activation job divergence (compiler_jobs.go)

Fix contentOverride lifetime (compiler_string_api.go)

Fix security scanning in wasm (compiler_orchestrator_engine.go)

Add parser.ReadFile wrapper (virtual_fs.go)

Add WorkflowIdentifier to wasm binary (cmd/gh-aw-wasm/main.go)

Test fixtures (20 total)

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

pelikhan left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pelikhan left a comment

Choose a reason for hiding this comment

Uh oh!

pelikhan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

Mossaka commented Feb 19, 2026 •

edited

Loading

Fix pre_activation job divergence (`compiler_jobs.go`)

Fix contentOverride lifetime (`compiler_string_api.go`)

Fix security scanning in wasm (`compiler_orchestrator_engine.go`)

Add parser.ReadFile wrapper (`virtual_fs.go`)

Add WorkflowIdentifier to wasm binary (`cmd/gh-aw-wasm/main.go`)

pelikhan left a comment •

edited

Loading