ci: capture compile stderr in parity + compile-smoke jobs#239
Merged
proggeramlug merged 1 commit intoApr 28, 2026
Merged
Conversation
…il diag)
Long-tail problem: 11 tests are tracked as `ci-env` in
test-parity/known_failures.json with the same vague reason — "macOS-14 CI
runner SDK/linker version interaction" — but no actual error message has
ever been pinned down because both compile harnesses silently discard
stderr:
- `run_parity_tests.sh:218` captures `compile_output` but only emits
"compile error" with no detail.
- `.github/workflows/test.yml::compile-smoke` redirects with `2>/dev/null`,
so the compile-smoke job summary lists only the failing test names.
Result: every PR that adds a Buffer/Uint8Array regression test ends up in
the same skip list ("ci-env / passes-locally") on inference alone, and the
underlying root cause never gets fixed.
Two changes, both diagnostic-only:
1. `run_parity_tests.sh` — on compile-fail, write the captured
`compile_output` to `test-parity/output/<test_name>.compile_error.log`.
Existing flow unchanged; no SKIP_TESTS / known_failures.json edits.
2. `.github/workflows/test.yml` — two new artifact upload steps:
- `parity-compile-errors-${{ runner.os }}` from the parity job
(gathers the per-test logs the script now writes).
- `compile-smoke-error-logs` from compile-smoke, with the per-test
stderr captured to `/tmp/perry_smoke_logs/`. The compile-smoke loop
also now `head -n 30`'s each failure into the job output so a quick
scan reveals the underlying error without downloading the artifact.
Both artifacts are uploaded with `if: always()` and
`if-no-files-found: ignore`, so when CI is green there's no artifact noise.
Once the actual link error is captured (next CI run on this PR will
trigger the parity job's stderr-capture path for the existing `ci-env`
tests), a follow-up PR can either bump the runner to macos-15 (the
maintainer's own hypothesis in v0.5.354 changelog) or fix the root cause
in Perry — depending on what the artifact shows.
No source changes, no SKIP_TESTS edits, no version bump. Pure diagnostic
plumbing. Once the long-tail is closed, these capture paths stay as
defensive infra for future regressions.
TheHypnoo
added a commit
to TheHypnoo/perry
that referenced
this pull request
Apr 28, 2026
`crates/perry-codegen/src/expr.rs` and `lower_call.rs` emit `call void @llvm.assume(i1 ...)` six times for branchless Buffer bounds checks (the v0.5.183 PerryTS#92 numeric-read intrinsic + the v0.5.190 small-buffer slab fast path), but the intrinsic was never declared in the LLVM IR module's prelude. The math intrinsics (`llvm.sqrt.f64`, `llvm.floor.f64`, etc.) are declared at runtime_decls.rs:117-121 — assume was simply missed. Apple Clang ≥21 (Xcode 26 — what local devs run) auto-recognises the intrinsic even when undeclared in the IR. Apple Clang 15 (LLVM 17 — what ships on the macos-14 GitHub runner via Xcode 15.x) requires the explicit declaration and errors out: /var/folders/.../perry_llvm.ll:1281:13: error: use of undefined value '@llvm.assume' call void @llvm.assume(i1 %r43) 1 error generated. This was the actual root cause behind the long-tail of `ci-env` Buffer/typed-array test skips in `test-parity/known_failures.json`. The diagnosis path: 1. The compile-stderr capture artifacts added in PR PerryTS#239 preserved the actual error message for the first time. 2. Downloading `parity-compile-errors-macOS` from a CI run revealed all 11 failing tests share the exact same `@llvm.assume` undefined-value error — not the speculated SDK/linker version interaction. Fix: one-line `module.declare_function("llvm.assume", VOID, &[I1])` in `runtime_decls.rs`, alongside the existing math-intrinsic declarations. Cleanup follow-on (now that the root cause is fixed): - `.github/workflows/test.yml::compile-smoke` — remove the 11-entry Buffer-family `SKIP_TESTS` list (test_gap_buffer_ops, test_buffer_*, test_inline_uint8array_param, test_issue_167_*, test_issue_227_*, test_gap_fetch_response, test_stress_*). UI/timer skips stay (different reasons). - `test-parity/known_failures.json` — remove 8 `ci-env` entries (test_gap_buffer_ops, test_gap_node_crypto_buffer, test_gap_typed_arrays, test_stress_buffer, test_buffer_small_alloc, test_buffer_numeric_read_intrinsic, test_issue_227_array_buffer_bytes, test_gap_fetch_response) plus 2 `bug` entries that were misdiagnosed but actually shared the same llvm.assume cause (test_inline_uint8array_param, test_issue_167_loop_alloca_stack_eat). test_stress_int_ops keeps its `bug` entry (separate console.log array-of-numbers padding bug exists independently). Verified locally on Apple Clang 21: all 10 previously-CI-failing Buffer/typed-array tests now compile cleanly. Stack: cargo build --release -p perry-codegen + perry; `./target/release/perry compile test-files/test_buffer_small_alloc.ts -o /tmp/...` produces a 0.7 MB binary; `grep "@llvm.assume" /tmp/perry_llvm_*.ll` confirms the IR now contains `declare void @llvm.assume(i1)`. Refs PR PerryTS#239 (the diagnostic capture infra that revealed this). # Conflicts: # .github/workflows/test.yml # test-parity/known_failures.json
proggeramlug
pushed a commit
that referenced
this pull request
Apr 28, 2026
`crates/perry-codegen/src/expr.rs` and `lower_call.rs` emit `call void @llvm.assume(i1 ...)` six times for branchless Buffer bounds checks (the v0.5.183 #92 numeric-read intrinsic + the v0.5.190 small-buffer slab fast path), but the intrinsic was never declared in the LLVM IR module's prelude. The math intrinsics (`llvm.sqrt.f64`, `llvm.floor.f64`, etc.) are declared at runtime_decls.rs:117-121 — assume was simply missed. Apple Clang ≥21 (Xcode 26 — what local devs run) auto-recognises the intrinsic even when undeclared in the IR. Apple Clang 15 (LLVM 17 — what ships on the macos-14 GitHub runner via Xcode 15.x) requires the explicit declaration and errors out: /var/folders/.../perry_llvm.ll:1281:13: error: use of undefined value '@llvm.assume' call void @llvm.assume(i1 %r43) 1 error generated. This was the actual root cause behind the long-tail of `ci-env` Buffer/typed-array test skips in `test-parity/known_failures.json`. The diagnosis path: 1. The compile-stderr capture artifacts added in PR #239 preserved the actual error message for the first time. 2. Downloading `parity-compile-errors-macOS` from a CI run revealed all 11 failing tests share the exact same `@llvm.assume` undefined-value error — not the speculated SDK/linker version interaction. Fix: one-line `module.declare_function("llvm.assume", VOID, &[I1])` in `runtime_decls.rs`, alongside the existing math-intrinsic declarations. Cleanup follow-on (now that the root cause is fixed): - `.github/workflows/test.yml::compile-smoke` — remove the 11-entry Buffer-family `SKIP_TESTS` list (test_gap_buffer_ops, test_buffer_*, test_inline_uint8array_param, test_issue_167_*, test_issue_227_*, test_gap_fetch_response, test_stress_*). UI/timer skips stay (different reasons). - `test-parity/known_failures.json` — remove 8 `ci-env` entries (test_gap_buffer_ops, test_gap_node_crypto_buffer, test_gap_typed_arrays, test_stress_buffer, test_buffer_small_alloc, test_buffer_numeric_read_intrinsic, test_issue_227_array_buffer_bytes, test_gap_fetch_response) plus 2 `bug` entries that were misdiagnosed but actually shared the same llvm.assume cause (test_inline_uint8array_param, test_issue_167_loop_alloca_stack_eat). test_stress_int_ops keeps its `bug` entry (separate console.log array-of-numbers padding bug exists independently). Verified locally on Apple Clang 21: all 10 previously-CI-failing Buffer/typed-array tests now compile cleanly. Stack: cargo build --release -p perry-codegen + perry; `./target/release/perry compile test-files/test_buffer_small_alloc.ts -o /tmp/...` produces a 0.7 MB binary; `grep "@llvm.assume" /tmp/perry_llvm_*.ll` confirms the IR now contains `declare void @llvm.assume(i1)`. Refs PR #239 (the diagnostic capture infra that revealed this). # Conflicts: # .github/workflows/test.yml # test-parity/known_failures.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
11 tests are tracked in
test-parity/known_failures.jsonwithstatus: \"ci-env\"and the same vague reason — "macOS-14 CI runner SDK/linker version interaction":test_gap_buffer_ops,test_buffer_small_alloc,test_buffer_numeric_read_intrinsic,test_gap_node_crypto_buffer,test_gap_typed_arrays,test_stress_buffer,test_inline_uint8array_param,test_issue_167_loop_alloca_stack_eat,test_issue_227_array_buffer_bytes,test_gap_fetch_response, plus the newtest_issue_234_blob_methodsfrom fix(fetch): #234 real Blob with arrayBuffer/text/bytes/slice instance methods #238.But the actual error message has never been pinned down, because both compile harnesses silently discard stderr:
run_parity_tests.sh:218capturescompile_outputinto a local var but only logs\"compile error\"with no detail..github/workflows/test.yml::compile-smokecompiles with2>/dev/null, so the job summary lists only the failing test names.Net effect: every PR that adds a Buffer / Uint8Array regression test ends up in the same skip list ("ci-env / passes-locally") on inference alone, and the underlying root cause never gets fixed. The v0.5.354 changelog itself acknowledges:
…but that hypothesis has never been verified against an actual error message.
What this PR does
Pure diagnostic plumbing — no source changes, no SKIP_TESTS edits, no runner bump, no version bump.
1.
run_parity_tests.shOn compile-fail, write the captured
compile_outputtotest-parity/output/<test_name>.compile_error.log. The existing per-test loop is unchanged; this just persists the stderr the script was already capturing into a memory-only var.2.
.github/workflows/test.ymlTwo new artifact upload steps (both
if: always()+if-no-files-found: ignoreso green CI runs produce no artifact noise):parity-compile-errors-${{ runner.os }}from theparityjob — gathers the per-test logs the script now writes.compile-smoke-error-logsfromcompile-smoke— captures stderr per test to/tmp/perry_smoke_logs/, plus inlineshead -n 30of each failure log into the job output (so a quick scan in the GitHub UI reveals the error without needing to download the artifact).What you'll see on the next CI run
The
parityjob naturally exercises everyci-envtest (it doesn't have a SKIP_TESTS list — it tracks them viaknown_failures.jsoninstead). With this PR, that job will produce ~11*.compile_error.logfiles in theparity-compile-errors-macOSartifact, one per known failure. Reading any of them gives the actual link error.Decision tree once we have the data
runs-on: macos-14→macos-15and removing the skips.Refs
test_inline_uint8array_param— original ci-env entry from this family)