Add distributed typescript benchmark harness#4698
Merged
joshua-spacetime merged 5 commits intomasterfrom Mar 25, 2026
Merged
Conversation
joshua-spacetime
force-pushed
the
joshua/keynote-2-dist-bench
branch
2 times, most recently
from
a7f5baa to
91d526d
Compare
bradleyshep
approved these changes
Mar 24, 2026
joshua-spacetime
force-pushed
the
joshua/keynote-2-dist-bench
branch
from
934f0ad to
4134efb
Compare
2 tasks
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Apr 17, 2026
# Description of Changes Refresh this stale PR against current `master`. Several original items were already applied upstream or conflict with recent keynote-2 work (#4616, #4647, #4678, #4682, #4698, #4703, #4743, #4745, #4753, #4757), so those are dropped. What remains is the still-relevant subset, rebased onto the current file structures. **README.md:** - Use `pnpm run demo` in Quick Demo (consistency with pnpm workspace) - Add `--concurrency` and `--alpha` to demo options - Add `--` separator to `docker compose run` bench example - Fix hardware config punctuation (add comma before "OS:") - Remove redundant Quick Start section; replace with link to DEVELOP.md for prerequisites and CLI reference - Add symlink for license **DEVELOP.md:** - Use `pnpm run` throughout (demo, prep, bench) instead of `npm run` - Drop the `-- ` pass-through after `pnpm run bench` (not needed with pnpm; matches the `#4703` testing examples) - Add Rust to Prerequisites - Add explicit list of valid connector names (`convex`, `spacetimedb`, `bun`, `postgres_rpc`, `cockroach_rpc`, `sqlite_rpc`, `supabase_rpc`, `planetscale_pg_rpc`) - Update CLI reference defaults to match methodology (seconds: 1→10, concurrency: 10→50) - Condense `docker compose run` bench example to a single line with `--` separator; fix `npm prep` → `pnpm run prep` **src/opts.ts:** (CLI parsing moved here in `#4703`; original PR targeted the now-gone inline parsers in `cli.ts`/`demo.ts`) - `parseBenchOptions`: bench `--seconds` default `1` → `10` - `parseDemoOptions`: demo `--concurrency` default `10` → `50` **.env.example:** - Comment out `USE_DOCKER=1` and `SKIP_CONVEX=1` so demo defaults (convex, spacetimedb) work out of the box - Comment out `CONVEX_USE_SHARDED_COUNTER=1` (still a supported knob, just off by default) # Dropped as superseded by master - Rust Client README section tweaks (heading capitalization, `bottlnecked`/`then` typo fixes) — section was removed by `#4753` - Rename `SPACETIME_METRICS_ENDPOINT` → `USE_SPACETIME_METRICS_ENDPOINT` — master's `src/config.ts` still reads the original name - Connector-name fixes in examples (`sqlite` → `sqlite_rpc`, `postgres` → `postgres_rpc`) — already corrected on master # API and ABI breaking changes None. # Expected complexity level and risk **1** – Documentation and default-value changes. No functional changes to core logic. # Testing - [x] `pnpm install` in `templates/keynote-2/` succeeds - [x] `pnpm run bench --help` / `pnpm run demo --help` render with valid-connec
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of Changes
The current keynote-2 benchmarks pipelines operations via
MAX_INFLIGHT_PER_WORKERin order to simulate a large number of client connections while running the benchmark locally or on a single machine.This patch adds a distributed benchmark mode for
templates/keynote-2so explicit SpacetimeDB client connections can be spread across multiple machines without changing the existing single-process benchmark flow.This is a pure extension.
npm run benchand the currentsrc/core/runner.tspath remain intact. The new distributed path adds a small coordinator/generator/control-plane harness specifically for multi-machine ts client runs.bench-dist-coordinator,bench-dist-generator, andbench-dist-controlwere added--open-parallelism) to avoid a connection storm/stopped, the epoch result is flagged with an error so the run can be retried cleanlySee
DEVELOP.mdfor instructions on how to run.API and ABI breaking changes
N/A
Expected complexity level and risk
3
Testing
Manual