Conversation
Co-authored-by: Asjas <3828967+Asjas@users.noreply.github.com> Agent-Logs-Url: https://github.com/Asjas/scratchyjs/sessions/03c930e4-7803-4161-9a55-3df11460248a
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Co-authored-by: Asjas <3828967+Asjas@users.noreply.github.com> Agent-Logs-Url: https://github.com/Asjas/scratchyjs/sessions/03c930e4-7803-4161-9a55-3df11460248a
Copilot
AI
changed the title
[WIP] Add performance benchmarking suite to documentation
feat: performance benchmarking suite, CI comment report, and template tsconfig fix
Mar 23, 2026
Contributor
|
📚 Docs preview deployed https://scratchyjs-docs-pr-36.asjas.workers.dev Updates automatically on every push to this PR. |
Contributor
Coverage Report
File CoverageNo changed files found. |
Contributor
⚡ Benchmark Results
|
| Suite — Benchmark | ops/sec | mean (µs) | p99 (µs) | ±rme |
|---|---|---|---|---|
| SharedRingBuffer – small payload (64 B) — write 64 bytes | 518.9 K | 0.0019 | 0.0075 | ±1.44% |
| SharedRingBuffer – small payload (64 B) — write + read 64 bytes | 467.0 K | 0.0021 | 0.0078 | ±1.42% |
| SharedRingBuffer – medium payload (1 KB) — write 1 KB | 244.4 K | 0.0041 | 0.0142 | ±0.58% |
| SharedRingBuffer – medium payload (1 KB) — write + read 1 KB | 210.0 K | 0.0048 | 0.0215 | ±1.33% |
| SharedRingBuffer – large payload (16 KB) — write 16 KB | 57.5 K | 0.0174 | 0.0424 | ±1.56% |
| SharedRingBuffer – large payload (16 KB) — write + read 16 KB | 83.8 K | 0.0119 | 0.0420 | ±18.76% |
| SharedRingBuffer – sequential throughput (100 × 64 B) — 100 write + read cycles | 40.7 K | 0.0246 | 0.0441 | ±0.39% |
| SharedRingBuffer – introspection — availableToRead | 14.47 M | 0.0001 | 0.0001 | ±0.10% |
| SharedRingBuffer – introspection — isEmpty | 14.88 M | 0.0001 | 0.0001 | ±0.10% |
| SharedRingBuffer – introspection — isFull | 14.74 M | 0.0001 | 0.0001 | ±0.21% |
benchmarks/renderer/shared-buffer.bench.ts
| Suite — Benchmark | ops/sec | mean (µs) | p99 (µs) | ±rme |
|---|---|---|---|---|
| SharedBuffer – allocation — createSharedBuffer(4 KB) | 595.6 K | 0.0017 | 0.0070 | ±1.15% |
| SharedBuffer – allocation — createSharedBuffer(64 KB) | 43.3 K | 0.0231 | 0.0426 | ±0.75% |
| SharedBuffer – small payload round-trip — write small JSON | 314.4 K | 0.0032 | 0.0120 | ±0.43% |
| SharedBuffer – small payload round-trip — write + read small JSON | 246.4 K | 0.0041 | 0.0135 | ±0.49% |
| SharedBuffer – medium payload round-trip — write medium JSON (~2 KB) | 99.7 K | 0.0100 | 0.0261 | ±0.40% |
| SharedBuffer – medium payload round-trip — write + read medium JSON (~2 KB) | 57.1 K | 0.0175 | 0.0390 | ±0.94% |
| SharedBuffer – large payload round-trip — write large JSON (~10 KB) | 15.5 K | 0.0647 | 0.0886 | ±0.31% |
| SharedBuffer – large payload round-trip — write + read large JSON (~10 KB) | 7.7 K | 0.1293 | 0.1658 | ±0.35% |
benchmarks/utils/ip-address.bench.ts
| Suite — Benchmark | ops/sec | mean (µs) | p99 (µs) | ±rme |
|---|---|---|---|---|
| getClientIPAddress – no IP headers — no IP-related headers → null | 2.55 M | 0.0004 | 0.0006 | ±0.18% |
| getClientIPAddress – single header — cf-connecting-ip (Cloudflare) | 1.68 M | 0.0006 | 0.0010 | ±0.34% |
| getClientIPAddress – single header — x-forwarded-for (simple) | 1.97 M | 0.0005 | 0.0007 | ±0.28% |
| getClientIPAddress – single header — x-real-ip | 1.47 M | 0.0007 | 0.0011 | ±0.29% |
| getClientIPAddress – single header — true-client-ip (Akamai / Cloudflare Enterprise) | 1.54 M | 0.0006 | 0.0010 | ±0.19% |
| getClientIPAddress – x-forwarded-for multi-hop — 2-hop chain | 1.78 M | 0.0006 | 0.0008 | ±3.06% |
| getClientIPAddress – x-forwarded-for multi-hop — 4-hop chain | 1.70 M | 0.0006 | 0.0009 | ±1.13% |
| getClientIPAddress – Forwarded header (RFC 7239) — simple for= directive | 991.0 K | 0.0010 | 0.0015 | ±1.33% |
| getClientIPAddress – Forwarded header (RFC 7239) — for= with port | 749.7 K | 0.0013 | 0.0020 | ±0.35% |
| getClientIPAddress – Forwarded header (RFC 7239) — IPv6 literal | 761.4 K | 0.0013 | 0.0017 | ±0.27% |
| getClientIPAddress – Forwarded header (RFC 7239) — multi-hop Forwarded | 834.8 K | 0.0012 | 0.0017 | ±1.65% |
| getClientIPAddress – IPv6 addresses — x-forwarded-for IPv6 | 1.14 M | 0.0009 | 0.0010 | ±0.28% |
benchmarks/utils/promise.bench.ts
| Suite — Benchmark | ops/sec | mean (µs) | p99 (µs) | ±rme |
|---|---|---|---|---|
| promiseHash – concurrent resolution — 2 already-resolved promises | 869.3 K | 0.0012 | 0.0031 | ±1.03% |
| promiseHash – concurrent resolution — 5 already-resolved promises | 524.5 K | 0.0019 | 0.0034 | ±1.72% |
| promiseHash – concurrent resolution — 10 already-resolved promises | 311.2 K | 0.0032 | 0.0055 | ±1.57% |
| promiseHash – concurrent resolution — 5 promises with object values | 485.4 K | 0.0021 | 0.0026 | ±0.27% |
| timeout – wrapping fast promises — timeout wrapping an already-resolved promise (1 s budget) | 965.2 K | 0.0010 | 0.0015 | ±0.34% |
| timeout – wrapping fast promises — timeout wrapping an already-resolved object (5 s budget) | 897.6 K | 0.0011 | 0.0017 | ±0.31% |
benchmarks/utils/safe-redirect.bench.ts
| Suite — Benchmark | ops/sec | mean (µs) | p99 (µs) | ±rme |
|---|---|---|---|---|
| safeRedirect – valid paths — root path / | 6.89 M | 0.0001 | 0.0002 | ±0.12% |
| safeRedirect – valid paths — simple path /dashboard | 3.47 M | 0.0003 | 0.0003 | ±0.11% |
| safeRedirect – valid paths — nested path /settings/profile | 3.11 M | 0.0003 | 0.0004 | ±0.10% |
| safeRedirect – valid paths — path with query string /search?q=hello | 3.38 M | 0.0003 | 0.0005 | ±0.10% |
| safeRedirect – valid paths — path with hash /docs#section | 3.46 M | 0.0003 | 0.0003 | ±0.10% |
| safeRedirect – rejected inputs — absolute URL https://evil.com | 4.67 M | 0.0002 | 0.0004 | ±0.10% |
| safeRedirect – rejected inputs — protocol-relative URL //evil.com | 4.62 M | 0.0002 | 0.0002 | ±0.09% |
| safeRedirect – rejected inputs — backslash-relative /\evil.com | 4.63 M | 0.0002 | 0.0002 | ±0.09% |
| safeRedirect – rejected inputs — path traversal /../etc/passwd | 4.07 M | 0.0002 | 0.0003 | ±0.11% |
| safeRedirect – rejected inputs — null input | 16.04 M | 0.0001 | 0.0001 | ±0.09% |
| safeRedirect – rejected inputs — undefined input | 15.95 M | 0.0001 | 0.0001 | ±0.11% |
| safeRedirect – rejected inputs — empty string | 15.82 M | 0.0001 | 0.0001 | ±0.17% |
| safeRedirect – percent-encoded bypass — percent-encoded // (%2F%2F) | 4.40 M | 0.0002 | 0.0004 | ±0.24% |
| safeRedirect – percent-encoded bypass — percent-encoded path traversal (%2e%2e) | 4.79 M | 0.0002 | 0.0002 | ±0.10% |
| safeRedirect – percent-encoded bypass — mixed percent-encoded absolute URL | 4.45 M | 0.0002 | 0.0003 | ±0.13% |
| safeRedirect – custom default redirect — valid path with custom default | 3.56 M | 0.0003 | 0.0003 | ±0.25% |
| safeRedirect – custom default redirect — invalid input with custom default | 4.79 M | 0.0002 | 0.0002 | ±0.10% |
ℹ️ No base results from
mainfound — delta column omitted on first run.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a Vitest-powered benchmarking suite for key framework hot paths, wires it into CI with an automated PR comment report, and fixes root tsconfig.json exclusions to avoid template-related typecheck noise.
Changes:
- Added
benchmarks/suites plus a dedicatedvitest.bench.config.tsandpnpm bench/pnpm bench:ciscripts. - Introduced a new GitHub Actions workflow to run benchmarks and post/update a PR comment with results (and deltas vs
mainwhen available). - Updated docs/README/sidebar and excluded the create-scratchy-app template directory from the root tsconfig.
Reviewed changes
Copilot reviewed 12 out of 14 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
vitest.bench.config.ts |
Dedicated Vitest config to run .bench.ts files and emit JSON results. |
package.json |
Adds bench and bench:ci scripts for local and CI runs. |
benchmarks/renderer/ring-buffer.bench.ts |
Bench suite for SharedRingBuffer write/read throughput and getters. |
benchmarks/renderer/shared-buffer.bench.ts |
Bench suite for SharedBuffer allocation and JSON round-trips. |
benchmarks/utils/safe-redirect.bench.ts |
Bench suite for safeRedirect happy paths and rejection fast-paths. |
benchmarks/utils/promise.bench.ts |
Bench suite for promiseHash and timeout overhead. |
benchmarks/utils/ip-address.bench.ts |
Bench suite for getClientIPAddress across common header patterns. |
.github/workflows/benchmarks.yml |
New CI workflow to run benchmarks, upload artifacts, and post PR comment report. |
docs/benchmarks.md |
Reference documentation for running/adding benchmarks and interpreting output. |
docs/.vitepress/config.ts |
Adds Benchmarks page to the VitePress sidebar. |
README.md |
Adds a Benchmarks section and links to docs. |
.gitignore |
Ignores generated benchmarks/results.json. |
tsconfig.json |
Excludes create-scratchy-app template directory to avoid false-positive type errors. |
IMPLEMENTATION_PLAN.md |
Marks the benchmarking suite item complete. |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: A-J Roos <asjasroos@pm.me>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: A-J Roos <asjasroos@pm.me>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the performance benchmarking suite from the implementation plan, wires it into CI with a PR comment report mirroring the coverage-report pattern, and fixes a long-standing root
tsconfig.jsongap that caused ~200 false-positive type errors.Benchmarks
Five Vitest bench suites under
benchmarks/:renderer/ring-buffer.bench.tsSharedRingBufferwrite/read at 64 B / 1 KB / 16 KB + sequential throughput + introspectionrenderer/shared-buffer.bench.tsSharedBufferallocation + JSON payload round-trips at ~100 B / ~2 KB / ~10 KButils/safe-redirect.bench.tssafeRedirectvalid paths, rejection fast-paths, percent-encoded bypass attemptsutils/promise.bench.tspromiseHash(2–10 entries) +timeoutwrapper overheadutils/ip-address.bench.tsgetClientIPAddressacross all header patterns including RFC 7239Forwardedand multi-hopx-forwarded-forNew scripts in
package.json:A dedicated
vitest.bench.config.tskeeps bench files out ofpnpm test.CI workflow (
.github/workflows/benchmarks.yml)Two jobs, same structure as
coverage-report:benchmark— runspnpm bench:cion Node 24, uploadsbenchmark-resultsartifact (30-day retention). Fires on push tomainand on PRs touchingbenchmarks/**,packages/**,vitest.bench.config.ts, orpnpm-lock.yaml.benchmark-report— PR-only. Downloads current + base (main) artifacts viaactions/download-artifact+dawidd6/action-download-artifact, then usesactions/github-scriptto post (or update) a single comment. Uses a hidden HTML marker to avoid comment spam on re-runs.Comment format — one collapsible
<details>block per file:main-13.0%+13.6%🚀Delta thresholds:⚠️ = >10% slower, 🚀 = >10% faster. Delta column is omitted on the first run (no base artifact on
mainyet).Docs
docs/benchmarks.md— reference page covering all suites, output columns, config, and how to add a new benchmarkREADME.mdgets a Benchmarks sectionIMPLEMENTATION_PLAN.mditem marked donetsconfig fix
packages/create-scratchy-app/src/template/imports external packages (@builder.io/qwik,drizzle-orm, etc.) that aren't installed in this monorepo — they're dependencies of the generated app. The per-package tsconfig already excludedsrc/template; the root one didn't, producing ~200 spurious errors when runningtsc --noEmitdirectly.✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.