Skip to content

Harden flaky native process-lifecycle tests for slow CI#465

Merged
dakra merged 1 commit into
mainfrom
fix-flaky-native-process-tests
Jun 24, 2026
Merged

Harden flaky native process-lifecycle tests for slow CI#465
dakra merged 1 commit into
mainfrom
fix-flaky-native-process-tests

Conversation

@dakra

@dakra dakra commented Jun 24, 2026

Copy link
Copy Markdown
Owner

Three native tests flake on the heavily-loaded GitHub runner, which runs make -j$(nproc) test-native — spawning a real PTY child per test file in parallel:

  • ghostel-test-exec-delete-process-kills-sighup-ignoring-child
  • ghostel-test-exec-kill-buffer-leaves-sighup-ignoring-child-live
  • ghostel-test-compile-kill-compilation-finds-live-buffer

Two distinct causes:

1. Regression in kill-compilation-finds-live-buffer

The readiness handshake added in 36d9510 ("Harden flaky tests") was dropped in 9e8460a ("Make ghostel buffers read-only"). Without it, kill-compilation (SIGINT) races process startup: the signal can land before /bin/sh -c "exec /bin/sleep 30" has exec'd sleep / the PTY foreground process group has settled, so finalize may not happen within the wait. Restore the GHOSTEL_KILL_READY marker and wait for it before signalling.

2. Tight 5s startup waits under parallel CI load

The two exec SIGHUP tests are logically sound (verified both PTY backends: kill-buffer → SIGHUP which the child traps, delete-process → SIGKILL), but rely on a hard-coded 5s startup wait. That's too tight under -j$(nproc) load, made worse now that the native PTY backend runs them twice via with-pty-matrix. The same 5s recurs across osc/exec/spawn/shell/keys native tests.

Fix centrally: add ghostel-test--timeout-scale (1 locally, 4 when CI is set, overridable via GHOSTEL_TEST_TIMEOUT_SCALE) and apply it in ghostel-test--wait-for / ghostel-test--wait-until. Every polling wait routes through these two, so the whole suite gets headroom in one place. The helpers early-exit on their predicate, so the larger ceiling costs nothing on passing runs. The scale gates on a positive numeric parse, so an empty or malformed override falls through to the CI/default logic instead of zeroing every timeout.

The unscaled fixed-window/negative assertions (accept-process-output proc 0.2, etc.) are deliberately left alone — scaling a "nothing should happen within this window" guard would change its meaning.

Verification

  • Three named tests pass locally (scale=1) and under simulated CI (CI=true, scale=4), both fast.
  • Scale resolution checked across unset / CI / empty / garbage / zero / valid / fractional overrides.
  • make -j8 all → exit 0, all suites green. The changed helper is a prerequisite of every test stamp, so the full elisp + native suite re-ran under the new code.

Two distinct causes made these flaky on the heavily-loaded GitHub
runner (which runs `make -j$(nproc) test-native', spawning a real PTY
child per test file in parallel):

1. Regression in `ghostel-test-compile-kill-compilation-finds-live-
   buffer'.  The earlier readiness handshake (commit 36d9510) was
   dropped in commit 9e8460a, so `kill-compilation' (SIGINT) raced
   process startup.  Restore the `GHOSTEL_KILL_READY' marker and wait
   for it before signalling, so SIGINT lands on a fully-exec'd sleep.

2. The two exec SIGHUP tests (and many other native tests) rely on a
   hard-coded 5s startup wait.  They are logically sound but too tight
   under parallel CI load, made worse now that the native PTY backend
   runs them twice via `with-pty-matrix'.  Add `ghostel-test--timeout-
   scale' (1 locally, 4 when CI is set, overridable via
   GHOSTEL_TEST_TIMEOUT_SCALE) and apply it in `ghostel-test--wait-for'
   and `ghostel-test--wait-until'.  Every polling wait routes through
   these two, so the whole suite gets headroom in one place.  The
   helpers early-exit on their predicate, so the larger ceiling costs
   nothing on passing runs.  The scale gates on a positive numeric
   parse so an empty or malformed override falls through instead of
   zeroing every timeout.
@dakra
dakra merged commit 25321dc into main Jun 24, 2026
24 checks passed
@dakra
dakra deleted the fix-flaky-native-process-tests branch June 24, 2026 16:40
@dakra
dakra temporarily deployed to github-pages June 24, 2026 16:40 — with GitHub Actions Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant