Live per-step domain events under @serenity-js/playwright-test — proposal for an eager-announce mode

[nbarrett]

Hi @jan-molak and team!

Following up on discussion #2797 — where the suggested "ConsoleReporter alternative via a custom StageCrewMember" was a great pointer, and I built NGX Ramblers' DomainEventPublisher on top of it.

That StageCrewMember design worked cleanly under WebdriverIO for about a year: InteractionFinished / TaskFinished events reached my crew member synchronously, and I streamed each step into an admin UI over WebSocket for live progress during long-running scenarios.

When we migrated to @serenity-js/playwright-test last week, the crew member went quiet. No errors — just no events until the browser closed, at which point every step arrived in a single burst. Reading SerenityReporterForPlaywrightTest.ts the reason became clear: PlaywrightEventBuffer collects everything per test and only flushes via this.serenity.announce(...events) at onTestEnd.

Architecturally that makes sense — Playwright's TestStep.duration / step.error / step.result are only final at step end, and buffering keeps Serenity's event construction clean. But for a consumer wanting live domain-event delivery during scenario execution, there's currently no StageCrewMember-based path under Playwright-test.

What I ended up doing

A plain Playwright Reporter registered alongside @serenity-js/playwright-test, tapping onStepEnd (filtering category === "test.step") and POSTing each step out-of-band. It works, but it sits outside Serenity's event model — I had to synthesise my own payload shape and lost the semantic DomainEvent typing.

Full write-up with diagrams here.

Suggestion

Would an opt-in eager announce mode on the @serenity-js/playwright-test reporter be feasible? Something like:

reporter: [
  ["@serenity-js/playwright-test", {
    crew: [/* ... */],
    announceEventsEagerly: true   // or emitPerStep: true
  }]
]

Semantics:

• At onStepEnd, if step.category === "test.step", synthesise the InteractionFinished / TaskFinished / related events and this.serenity.announce() them immediately
• At onTestEnd, announce only the scene-lifecycle events that genuinely need the full result (SceneFinished, retry summaries, etc.)
• PlaywrightEventBuffer would still back-fill anything that the live path couldn't emit (e.g. retries, crashes, deferred after-all hooks)

From a consumer's perspective this would restore the elegance of "one Serenity event pipeline, framework-agnostic" — a StageCrewMember like the one in #2797 could then serve both WDIO and Playwright-test setups unchanged.

I might be missing an existing hook that already offers this — if so, please point me at it and I'll happily update the article. Either way, two smaller asks:

1. A note in the @serenity-js/playwright-test API page or handbook explicitly calling out the end-of-test flush timing — I checked both and couldn't find it, which is how our silent failure went undiagnosed for an afternoon.
2. A cross-link from Alternative implementation of the ConsoleReporter #2797 noting the Playwright-test caveat, since that thread was my starting point in 2025.

Happy to help prototype a PR if there's appetite.

Thanks,
Nick (nbarrett/ngx-ramblers)

[jan-molak]

Hi Nick,

I really like where you’re going with this - especially the idea of real-time reporting and building on top of the Serenity/JS event stream. That feels like a very natural fit with the architecture, rather than layering something parallel on top.

One thought I had while reading your proposal: would you be open to a slightly different solution shape? Instead of buffering events on the Playwright side, what if we registered a WebSocket reporter automatically when Playwright Test initialises the Serenity/JS reporter?
At that point, we already have everything wired up, so rather than collecting events and flushing them later, we could stream them out as they happen.

Following that, the worker-level stage crew member could push events directly to the WebSocket server rather than writing to a buffer first. That would simplify things quite a bit and keep the flow aligned with how Serenity/JS is designed to work - events in, events out, no detours.

A bonus is that it opens up some interesting possibilities beyond reporting alone:

• piping the event stream into external services
• building live dashboards or progress views
• integrating with CI or observability tooling without extra adapters

Feels like this could give you the real-time behaviour you’re after while also reducing some of the moving parts.

Curious how that sits with you?
Jan

[nbarrett]

Hi Jan,

Apologies for the slow reply - yes, the shape you describe works for me. Rather than debate it in the abstract I've put together a spike along those lines so we have something concrete to poke at: #3395

The gist: an opt-in liveEvents: true option on the reporter config. When it's on, the reporter process starts a WebSocket server on an ephemeral localhost port, and each worker registers a WorkerEventStreamer crew member (sitting alongside the existing WorkerEventStreamWriter) that pushes each domain event out as it happens. The reporter announces them to its stage straight away, so reporter-side crew members get InteractionFinished / TaskFinished while the scenario is still running - which restores the "one event pipeline" behaviour I had under WebdriverIO.

The end-of-test flush from PlaywrightEventBuffer still runs, but skips anything already announced live, so each event goes out exactly once, and anything the live path couldn't deliver (crashed workers, deferred afterAll hooks) still gets back-filled. Events that arrive before the reporter has seen onTestBegin for their scene are held back and announced just after the scene-start events, so crew members never see an interaction before SceneStarts.

A few judgement calls you may well want to overrule:

• I've left the disk-based WorkerEventStreamWriter alone, so the WebSocket stream sits alongside it rather than replacing it. Happy to consolidate if you'd rather go stream-only.
• Dedup between the live stream and the buffered flush is keyed on the serialised event (constructor.name + toJSON()) rather than equals(), because events like ActorEntersStage carry plain serialised-actor objects that equals() compares by reference.
• With parallel workers, crew members see events from concurrent scenes interleaved, correlated by sceneId. The Serenity BDD reporter already routes by sceneId so that looked safe, but you know the crew better than I do.
• It's a plain boolean for now. It could grow into { port } or similar if you want external subscribers connecting directly, which would open up the dashboards and CI integrations you mentioned.

There's an integration test covering ordering, exactly-once delivery and actual liveness: two interactions separated by a two-second pause have to arrive more than a second apart, which fails under the current buffered behaviour.

Thanks again for the pointer - look forward to a little collaboration!

Nick