Skip to content

Replace JsInstance pool with single worker and FIFO queue#4663

Merged
joshua-spacetime merged 3 commits intomasterfrom
joshua/v8-perf
Mar 24, 2026
Merged

Replace JsInstance pool with single worker and FIFO queue#4663
joshua-spacetime merged 3 commits intomasterfrom
joshua/v8-perf

Conversation

@joshua-spacetime
Copy link
Collaborator

@joshua-spacetime joshua-spacetime commented Mar 18, 2026

Description of Changes

Before this change, JS reducer requests borrowed a JsInstance from a pool. If no idle instance was available, we created another instance, which meant another V8 worker thread. Under load, this meant reducers bouncing across multiple OS threads.

After this change, JS reducers go through a single long-lived JsInstance fed by a FIFO queue which results in much better cache locality. More accurately, each module now allocates a single OS thread, on which reducers (and most operations) run. Modules do not share workers/threads. And modules do not create multiple threads for running reducers.

Note, the original instance pool is still used for procedures. It should probably be bounded, but I didn't make any changes to it. It's also used for executing views during initial subscription to avoid a reentrancy deadlock. The latter should be fixed and moved over to the JS worker thread at some point.

API and ABI breaking changes

N/A

Expected complexity level and risk

4

Testing

NODE_OPTIONS="--max-old-space-size=8192" \
MAX_INFLIGHT_PER_WORKER=512 \
BENCH_PRECOMPUTED_TRANSFER_PAIRS=1000000 \
pnpm bench test-1 --seconds 10 --concurrency 50 --alpha 1.5 --connectors spacetimedb
50K TPS -> 85K TPS on m2 mac

@joshua-spacetime joshua-spacetime force-pushed the joshua/v8-perf branch 5 times, most recently from 18efc3d to fbceefc Compare March 19, 2026 01:00
@joshua-spacetime joshua-spacetime changed the title test: v8 perf ideas Replace JsInstance pool with single worker and FIFO queue Mar 19, 2026
@joshua-spacetime joshua-spacetime marked this pull request as ready for review March 19, 2026 01:02
@Centril
Copy link
Contributor

Centril commented Mar 19, 2026

On phoenix nap + rust client, this takes performance to about 150k TPS, which is around 15k lower than for the rust module.

Copy link
Collaborator

@coolreader18 coolreader18 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now

@joshua-spacetime joshua-spacetime added this pull request to the merge queue Mar 24, 2026
Merged via the queue into master with commit 05a4a7b Mar 24, 2026
35 of 36 checks passed
@joshua-spacetime joshua-spacetime deleted the joshua/v8-perf branch March 24, 2026 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants