Skip to content

AsyncQueuer leaks memory in Node.js via unintended retryer key propagation #198

@simonmeyerrr

Description

@simonmeyerrr

TanStack Pacer version

@tanstack/pacer@0.20.1 (current latest)

Framework/Library version

Node 22 / 24

Describe the bug and the steps to reproduce it

AsyncQueuer causes unbounded heap growth when used in a Node.js (non-browser) environment. Memory grows linearly with the number of items processed and is never reclaimed, even after forced GC.

In packages/pacer/src/async-queuer.ts, the execute() method creates an AsyncRetryer for every item:

const currentAsyncRetryer = new AsyncRetryer(this.fn, {
  ...this.options.asyncRetryerOptions,
  key: `${this.key}-retryer-${currentExecuteCount}`,
})

When the AsyncQueuer is created without a key (the common case — key is optional and intended for devtools), this.key is undefined. The template literal coerces undefined to the truthy string "undefined-retryer-1", "undefined-retryer-2", etc.

This truthy key triggers two things in each AsyncRetryer instance:

  1. Constructor (async-retryer.ts): The if (this.key) guard passes, so pacerEventClient.on(...) is called — registering a devtools event listener.
  2. Every #setState call (async-retryer.ts): emitChange('AsyncRetryer', this) is called, which checks if (!key) return — but the key is the truthy string "undefined-retryer-N", so it proceeds to pacerEventClient.emit(...).

In Node.js, the EventClient from @tanstack/devtools-event-client can never connect to a devtools bus (no window, no globalThis.__TANSTACK_EVENT_TARGET__). Each emit() call pushes the event payload (containing the full AsyncRetryer instance) into pacerEventClient.#queuedEvents — a module-level singleton array.

Each AsyncRetryer generates ~4-5 state-change events during its lifecycle. These accumulate in the global #queuedEvents array, pinning every retryer (and its closures over user data) in memory permanently.

Your Minimal, Reproducible Example - (Sandbox Highly Recommended)

Require node environment with --expose-gc to reproduce which is not possible in CodeSandbox or Stackblitz

Screenshots or Videos (Optional)

Scenario Heap after round 1 Heap after round 1,000 Growth
@tanstack/pacer@0.20.1 (no key, before fix) 6.7 MB 590.0 MB +583.3 MB
@tanstack/pacer@0.20.1 (no key, after fix) 5.7 MB 6.0 MB +0.2 MB
// reproduce-leak.ts — run with: npx tsx --expose-gc reproduce-leak.ts
import { AsyncQueuer } from '@tanstack/pacer'

const ROUNDS = 1000
const ITEMS_PER_ROUND = 100

async function main() {
  const queuer = new AsyncQueuer<number>(
    async (item) => item * 2,
    { concurrency: 5, started: false, throwOnError: false },
  )

  const heapHistory: number[] = []

  for (let round = 0; round < ROUNDS; round++) {
    for (let i = 0; i < ITEMS_PER_ROUND; i++) {
      queuer.addItem(round * ITEMS_PER_ROUND + i + 1)
    }
    queuer.start()

    while (queuer.store.state.items.length > 0 || queuer.store.state.activeItems.length > 0) {
      await new Promise((r) => setTimeout(r, 10))
    }
    queuer.stop()

    global.gc!()
    await new Promise((r) => setTimeout(r, 50))
    global.gc!()

    const heapMB = process.memoryUsage().heapUsed / 1024 / 1024
    heapHistory.push(heapMB)
    console.log(
      `Round ${String(round + 1).padStart(4)}: heap = ${heapMB.toFixed(1)} MB  (items processed: ${(round + 1) * ITEMS_PER_ROUND})`,
    )
  }

  const growth = heapHistory[heapHistory.length - 1]! - heapHistory[0]!
  console.log(`\nTotal heap growth: ${growth.toFixed(1)} MB over ${ROUNDS * ITEMS_PER_ROUND} items`)
}

main().catch(console.error)

Do you intend to try to help solve this bug with your own PR?

Yes, I am also opening a PR that solves the problem along side this issue

Terms & Code of Conduct

  • I agree to follow this project's Code of Conduct
  • I understand that if my bug cannot be reliable reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions