fix: normalize container pins in wasm golden tests#25750
Conversation
…CI job Agent-Logs-Url: https://github.com/github/gh-aw/sessions/b724d099-3e05-43d9-ac5d-c385d81a0d7c Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Fixes WASM golden test instability caused by environment-dependent Docker image digest pins (native compilation can load the action cache and append @sha256:..., but the WASM/Node path cannot), by normalizing/removing digest suffixes before comparison and updating goldens accordingly.
Changes:
- Added
normalizeContainerPins()(Go + JS) to strip@sha256:<64-hex>digest suffixes prior to golden comparisons. - Applied the new normalization in the Go WASM golden test path and the Node-based WASM golden runner.
- Regenerated golden fixtures to remove cache-dependent digest pins.
Show a summary per file
| File | Description |
|---|---|
| scripts/test-wasm-golden.mjs | Normalizes container digest pins (and heredoc delimiters) before comparing WASM output to Go-generated goldens. |
| pkg/workflow/wasm_golden_test.go | Normalizes container digest pins in the Go golden comparison to keep goldens stable across environments. |
| pkg/workflow/compiler.go | Adds shared normalization helper (normalizeContainerPins) alongside existing heredoc normalization utilities. |
| pkg/workflow/testdata/TestWasmGolden_CompileFixtures/with-imports.golden | Updates golden output to remove @sha256:... digest pins. |
| pkg/workflow/testdata/TestWasmGolden_CompileFixtures/basic-copilot.golden | Updates golden output to remove @sha256:... digest pins. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 5/5 changed files
- Comments generated: 0
🧪 Test Quality Sentinel ReportTest Quality Score: 100/100✅ Excellent test quality
Test Classification Details
Analysis✅
|
| Component | Score |
|---|---|
| Behavioral Coverage (design tests) | 40/40 |
| Error/Edge Case Coverage | 30/30 |
| Low Duplication | 20/20 |
| Proportional Growth | 10/10 |
| Total | 100/100 |
Language Support
Tests analyzed:
- 🐹 Go (
*_test.go): 1 modified test — unit (//go:build !integration) - 🟨 JavaScript (
*.test.cjs,*.test.js): 0 tests (changes toscripts/test-wasm-golden.mjsare in the helper script, not a test file)
Verdict
✅ Check passed. 0% of modified tests are implementation tests (threshold: 30%). The change is a targeted stability fix: adds container-pin normalization to golden-file comparisons in both Go and the companion JavaScript script, so tests pass regardless of whether the action cache is available during wasm compilation. No guideline violations detected.
📖 Understanding Test Classifications
Design Tests (High Value) verify what the system does:
- Assert on observable outputs, return values, or state changes
- Cover error paths and boundary conditions
- Would catch a behavioral regression if deleted
- Remain valid even after internal refactoring
Implementation Tests (Low Value) verify how the system does it:
- Assert on internal function calls (mocking internals)
- Only test the happy path with typical inputs
- Break during legitimate refactoring even when behavior is correct
- Give false assurance: they pass even when the system is wrong
Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.
🧪 Test quality analysis by Test Quality Sentinel · ● 554.8K · ◷
There was a problem hiding this comment.
✅ Test Quality Sentinel: 100/100. Test quality is excellent — 0% of modified tests are implementation tests (threshold: 30%). The single modified test (TestWasmGolden_CompileFixtures) is a high-value design test enforcing the behavioral contract that the compiler produces expected YAML output, with proper build tags, error handling, and no coding-guideline violations.
…ization pipeline Extract composite normalize helpers from the two-step normalizeContainerPins(normalizeHeredocDelimiters(...)) calls introduced in #25750. - Go: adds normalizeOutput(string) string in wasm_golden_test.go - JS: adds normalize(content) in scripts/test-wasm-golden.mjs Both helpers apply all normalization steps (heredoc delimiters + container pins) in one place, making future additions easier and reducing duplication. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The
build-wasmCI job fails because WASM golden tests compare against golden files generated by the Go test suite, which loads the repo's ActionCache (.github/aw/actions-lock.json) and pins Docker images with@sha256:…digests. The WASM binary in Node.js has no filesystem access to load the cache, so its output lacks digests.normalizeContainerPins()to strip@sha256:<hex>suffixes before golden comparison — mirrors the existingnormalizeHeredocDelimiters()patternwasm_golden_test.go) and JS (scripts/test-wasm-golden.mjs) test paths