Skip to content

Fixed flaky timeout-threshold tests racing real setTimeout#27690

Merged
9larsons merged 3 commits intomainfrom
tests/helper-timeout-fake-timers
May 5, 2026
Merged

Fixed flaky timeout-threshold tests racing real setTimeout#27690
9larsons merged 3 commits intomainfrom
tests/helper-timeout-fake-timers

Conversation

@9larsons
Copy link
Copy Markdown
Contributor

@9larsons 9larsons commented May 5, 2026

Summary

The {{#get}} and {{#recommendations}} "timeout threshold" tests asserted that the helper's setTimeout(threshold=1ms) fires before the API stub's setTimeout(5ms). That's a Node scheduler race, not a behavior assertion.

Switch each test to sinon.useFakeTimers({toFake: ['setTimeout', 'clearTimeout']}), kick off the helper, then clock.tickAsync(2) — fires the 1ms threshold timer but not the 5ms stub timer, so the timeout branch wins deterministically.

Why

Under CI load both timers can land in the same macrotask and the ordering becomes unreliable. Recent hits:

The pattern dates back to commit 270f288c48 for get.test.js; PR #27571 tried to harden the recommendations variant on 2026-04-27 by awaiting cachePartials, but that fixed setup, not the timer race.

toFake is scoped to setTimeout/clearTimeout so Date and other clocks aren't affected — the notify test in the same file still relies on real Date.now() and is unchanged (it isn't a race; it just measures elapsed time after the API resolves).

Test plan

  • Each rewritten suite passes locally 25/25 in a stress loop.
  • Full recommendations.test.js (4 passing) and get.test.js (47 passing) clean.
  • ESLint clean.

no ref

- {{#get}} and {{#recommendations}} timeout tests asserted that the
  helper's setTimeout(threshold=1ms) fired before the API stub's
  setTimeout(5ms), which is fundamentally a Node scheduler race
- under CI load, both timers can land in the same macrotask and the
  ordering becomes unreliable, e.g. recommendations.test.js:173 in
  run 25404408183 and get.test.js:562 in run 25399938361
- install sinon fake timers in each test, kick off the helper, then
  tickAsync(2) — that fires the 1ms threshold timer but not the 5ms
  API timer, so the timeout branch wins deterministically
- limited toFake to [setTimeout, clearTimeout] so Date and other
  timers aren't affected (the get notify test still uses real time)
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 5, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: dec14ab6-87b3-4914-a464-23f3dbd9c1a8

📥 Commits

Reviewing files that changed from the base of the PR and between 6444d5d and 67f53a6.

📒 Files selected for processing (2)
  • ghost/core/test/unit/frontend/helpers/get.test.js
  • ghost/core/test/unit/frontend/helpers/recommendations.test.js

Walkthrough

Two frontend helper unit tests were modified to use Sinon fake timers for deterministic timeout behavior. Timeout tests were rewritten to capture response promises before advancing a fake clock, use clock.tickAsync() to exceed configured thresholds, and then await responses. Tests assert timeout outcomes (aborted result or empty HTML) and error logging. Clock setup/teardown is performed using test lifecycle hooks or try/finally to restore the fake clock.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely identifies the main change: fixing flaky timeout tests by replacing real setTimeout with fake timers. It accurately describes the core problem and solution.
Description check ✅ Passed The description is directly related to the changeset, providing detailed context about the timeout race condition, the fix using sinon fake timers, and supporting evidence from CI failures.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch tests/helper-timeout-fake-timers

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

9larsons added 2 commits May 5, 2026 18:30
no ref

- replaced two-line "advance past threshold" comment with the
  single-line bound check that explains the magic number 2
@9larsons 9larsons enabled auto-merge (squash) May 5, 2026 23:46
@9larsons 9larsons merged commit d128c64 into main May 5, 2026
43 checks passed
@9larsons 9larsons deleted the tests/helper-timeout-fake-timers branch May 5, 2026 23:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant