Skip to content

browser runner: a failing test skips its own afterEach, cascading one failure into many #25

Description

@brainkim

Version: 0.2.18

In the browser runner, afterEach hooks run inside the try block, after await t.fn():

await t.fn();                                  // test body
for (const suite of [...chain].reverse()) {
  for (const hook of suite.afterEach) await hook();   // inside the try
}
...
} catch (e) { ...failed++ }

So when a test throws, its afterEach hooks are skipped entirely. Any per-test cleanup — restoring stubs, clearing DOM — silently doesn't happen, and the damage compounds into later tests.

Real case (crank #375, webkit): exactly one genuine failure existed, but 26 of the suite's files stub console.error in beforeEach and restore it in afterEach. The failing test skipped its restore, so every subsequent test that stubbed got Sinon's "Attempted to wrap error which is already wrapped", and the run reported 23 failures for 1 bug. The 22 phantom messages also buried the real failure's actual assertion text.

node:test and bun:test both run afterEach hooks even when the test fails (and report hook errors separately). The browser runner should match: run afterEach in a finally, keeping the test's own error as the reported failure if both throw.

(beforeEach failing → skipping the test body is fine and matches the other runtimes; it's only after-hooks that need the guarantee.)

Found while diagnosing the webkit failure now fixed on the crank side (bikeshaving/crank#379); related to the firefox work headed for 0.2.19 — would be good company in that release.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions