Skip to content

[Bug] Wall-clock timing assertions flake — Bun's setTimeout fires up to ~11 ms early on Windows' 15.625 ms tick #477

Description

@pathosDev

What happened

Three unit tests assert on wall-clock elapsed time and fail intermittently
in a full bun test run, which makes bun run test:coverage:gate unreliable —
the gate fails because bun test exits non-zero, not because of the coverage
floor (line coverage sits around 94 %, well above the 80 % floor).

Test Assertion
tests/unit/pattern/FuturesPatterns.test.tsafter > resolves with the factory value after the delay elapsed >= 25 for after(30, …)
tests/util/AsyncAssertions.test.tsassertDoesNotCompleteWithin > returns normally after ms when promise stays pending elapsed >= 25 for a 30 ms wait
tests/unit/pattern/FuturesPatterns.test.tsretry > exponential backoff respects maxDelayMs gap2 < 40 for a 30 ms capped delay (seen failing with 44)

Observed over roughly ten full runs on Windows: one or two fail per run, more
often with --coverage than without. Each passes reliably when its file is run
alone, and it reproduces on an unmodified baseline (git stash), so no recent
change causes it.

Root cause — measured, and it is not Date.now() granularity. The initial
suspicion was Windows' 15.6 ms clock granularity, but under Bun 1.3.1 on
Windows 11 Date.now() ticks at 1 ms and performance.now() at ~0.0001 ms.
The coarse quantum shows up in the timer scheduler, not the clock: Bun's
event loop decides a timer is due on the Windows 15.625 ms tick boundary, so a
setTimeout whose deadline sits just below a tick multiple can fire a full
tick early
. Measured (idle machine, performance.now() deltas):

setTimeout(20)  n=1500   min= 19.82   p50=31.13   max= 63.78     (never early)
setTimeout(30)  n=1500   min= 20.05   p50=30.52   max= 46.32     (up to ~10 ms early)
setTimeout(30)  n=2000   min= 18.67   p50=30.52   max=174.97     (up to ~11 ms early)
setTimeout(50)  n=1500   min= 54.04   p50=62.58   max= 72.06     (never early)

30 ms is the pathological value: it sits just under 2 × 15.625 = 31.25, so the
loop can treat it as due one tick in, at ~15.6–18.7 ms of real time. The
theoretical floor is nominal − 15.625, i.e. ~14.4 ms for a 30 ms timer — a
>= 25 bound has no margin at all and fails outright, no load required. That
also explains why AsyncAssertions.test.ts flakes even though it already uses
performance.now()
: switching clocks cannot fix an early-firing timer.

The retry case is the mirror image — late firing. A 30 ms setTimeout was
measured at up to 46 ms idle and 201 ms under CPU load, so the gap2 < 40
upper bound cannot survive; no tolerance value both permits real jitter and
still distinguishes the capped 30 ms delay from an uncapped 40 ms one.

Not a duplicate of #290: that issue is about resource contention under parallel
execution (port collisions, filesystem races, socket exhaustion) in the
multi-node and filesystem suites. These three fail purely on timer
quantization, reproduce with a single test file, and need no parallelism.

What did you expect

bun run test:coverage:gate is deterministic: timing-sensitive tests either
assert relative/virtual time, or use a wall-clock bound with real margin
against the platform's 15.625 ms timer quantum.

Reproduction

// Fails ~1 in 200 on an idle Windows machine, far more often under
// `bun test --coverage` load — no actor-ts code involved.
const nominalMs = 30;
for (let i = 0; i < 2000; i++) {
  const start = performance.now();
  await new Promise((resolve) => setTimeout(resolve, nominalMs));
  const elapsed = performance.now() - start;
  if (elapsed < 25) throw new Error(`setTimeout(30) fired after ${elapsed.toFixed(2)}ms`);
}

Or run the offenders directly:

bun test tests/unit/pattern/FuturesPatterns.test.ts tests/util/AsyncAssertions.test.ts --coverage

actor-ts version

0.11.0 (commit 42fa0fa)

Runtime

Bun

Runtime version

Bun 1.3.1 on Windows 11 (10.0.26200). The same class of skew is documented at
tests/unit/coordination/InMemoryLease.test.ts:84 for GitHub's hosted runners,
where it was estimated at 1–2 ms — an order of magnitude too small.

Proposed fix

Per test, not one blanket rule:

  • after / assertDoesNotCompleteWithin — the point is "a real timer
    elapsed", so measure with performance.now() and put the lower bound a full
    timer quantum below nominal (10 ms for a 30 ms delay: ~4 ms below the
    theoretical floor, ~9 ms below the measured minimum). Add the deterministic
    assertion that actually pins the contract — the factory is not invoked
    synchronously — instead of leaning on the clock for it. Keep the nominal
    delays as they are; raising them would slow the suite without removing the
    quantization.
  • retry backoff — make it wall-clock free. Add an optional
    sleep?: (ms: number) => Promise<void> seam to RetryOptions (mirroring the
    existing random?: () => number override on BackoffPolicy) and drive the
    test off ManualScheduler's virtual clock, so the schedule is asserted
    exactly: attempts at virtual 0, 20, 50 ms — proving the third delay is capped
    at 30 ms rather than 40 ms.
  • Record the quantum rationale once, in a shared test helper, so the next
    timing test does not re-derive it.

tests/unit/coordination/InMemoryLease.test.ts:95 (>= 80 for 2 × 50 ms) was
checked and left alone — setTimeout(50) never fires early, and two chained
50 ms sleeps measured a 120 ms minimum, so that bound has 40 ms of real margin.

Acceptance

  • bun run test:coverage:gate green across five consecutive runs.
  • bun run typecheck + bun test green.
  • No change to the coverage floor.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority: mediumUseful, not urgent

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions