Skip to content

Serializing an AbortSignal into a step adds ~0.5s fixed latency per step invocation on world-local #2795

Description

@AndrewBarba

Summary

Passing an AbortSignal (or AbortController) into a step's arguments adds ~0.5 s of fixed latency to every invocation of that step on world-local — a ~7× slowdown for a trivial step (84 ms → ~600 ms). The cost is entirely on the step side: creating the controller in the workflow is free, and the same workflow with the signal omitted from the step input runs at full speed.

On a world that has accumulated stream chunks, the same signal-bearing step degrades far past the fixed cost — see the companion issue about the world-local streamer's polling (readdir-per-tick) behavior, filed separately.

This makes the documented cancellation pattern (durable AbortController, signal threaded into long-running steps) unusable in practice on world-local: we had to pull it out of a feature branch after a 100-turn session e2e went from ~0.9 s/turn to a 240 s replay timeout.

Versions observed

  • @workflow/core@5.0.0-beta.26
  • @workflow/world-local@5.0.0-beta.22 (via createLocalWorld; in-process handler, WORKFLOW_TURBO=0)
  • Node v24, macOS (also reproduced on Linux CI)

Numbers

20 sequential trivial steps (append one line to a file, return):

WITHOUT signal: total=4025ms  per-step deltas ≈ 84ms
WITH signal:    total=13026ms per-step deltas ≈ 600ms   (~+515ms per step, flat)

Same shape inside our product's turn loop: per-turn latency 385 ms → 910 ms the moment the turn's one step carries a signal; bisected across five variants (hook only: free; controller created but signal not passed: free; signal passed: +~520 ms).

Repro

import { appendFile } from "node:fs/promises";

export async function timedStep(input: {
  logPath: string;
  index: number;
  abortSignal?: AbortSignal;
}) {
  "use step";
  await appendFile(input.logPath, `step ${input.index} ${Date.now()}\n`);
  return input.index;
}

export async function signalCostWorkflow(input: {
  count: number;
  withSignal: boolean;
  logPath: string;
}) {
  "use workflow";
  const controller = new AbortController();
  for (let index = 0; index < input.count; index++) {
    await timedStep({
      abortSignal: input.withSignal ? controller.signal : undefined,
      index,
      logPath: input.logPath,
    });
  }
  return "done";
}

Run once with withSignal: false and once with true; diff the timestamps.

Where the time appears to go

The serialized signal carries { streamName, hookToken }. On every step revival, the step host opens a tail reader on the abort stream for the step's entire duration (the real-time abort delivery path — gx(...) in the serialization chunk: new ox(runId, streamName).getReader() raced against a per-step teardown controller, pushed into the step context's ops). The ~0.5 s fixed cost tracks that reader's open/tail/teardown lifecycle against a stream that has no chunks and, in the common case, never will.

Two adjacent observations:

  • The controller's abrt_… system hook is registered per controller and never disposed, so long-lived sessions creating one controller per unit of work accumulate live system hooks.
  • The reader polls the chunk store every 100 ms with a full directory listing — that is the scaling half of this problem, filed separately with numbers.

Expected

A signal in step arguments should cost roughly what a hook read costs — ideally near-zero when never aborted: e.g. subscribe lazily/in-process first, or don't tie a polling stream tail's lifecycle to every step invocation.

Actual

Every invocation of a signal-bearing step pays ~0.5 s on world-local even in a fresh world, growing without bound as the world accumulates stream chunks.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions