Skip to content

fix(benchmarks): run SSR setup imports natively#7579

Merged
schiller-manuel merged 8 commits into
mainfrom
fix-ssr-codspeed-benchmark
Jun 7, 2026
Merged

fix(benchmarks): run SSR setup imports natively#7579
schiller-manuel merged 8 commits into
mainfrom
fix-ssr-codspeed-benchmark

Conversation

@Sheraff
Copy link
Copy Markdown
Collaborator

@Sheraff Sheraff commented Jun 7, 2026

Summary

  • run SSR perf benchmarks with NODE_ENV=production
  • import built SSR server entries via native file URLs so Vitest/Vite does not analyze the built server entry during setup
  • keep the setup import outside the measured benchmark body

Notes

  • Vite recognizes /* @vite-ignore */ in import analysis and recommends it in its dynamic import warning for intentionally opaque imports. It suppresses analysis/warnings for this Node-only native import path.

Test Plan

  • CI=1 NX_DAEMON=false WITH_INSTRUMENTATION=1 pnpm nx run @benchmarks/ssr:test:perf:react --outputStyle=stream --skipRemoteCache
  • CI=1 NX_DAEMON=false pnpm nx run @benchmarks/ssr:test:perf:solid --outputStyle=stream --skipRemoteCache
  • CI=1 NX_DAEMON=false pnpm nx run @benchmarks/ssr:test:perf:vue --outputStyle=stream --skipRemoteCache
  • CI=1 NX_DAEMON=false pnpm nx run @benchmarks/ssr:test:types --outputStyle=stream --skipRemoteCache

Summary by CodeRabbit

  • Chores

    • Benchmark run scripts now execute in production mode for more accurate perf results.
    • Benchmarking tooling dependency updated to a newer release.
  • Refactor

    • Benchmarks now load server entries via stable module URLs for reliable dynamic imports.
    • Initialization moved to module load (eager handler loading); explicit per-run setup/teardown and handler resets removed.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Jun 7, 2026

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Bench scripts now run vitest benches under NODE_ENV=production. React, Solid, and Vue SSR benchmarks compute absolute module URLs from import.meta.url and eagerly import the server bundle default (StartRequestHandler) at module scope, removing setup/teardown and suite hooks.

Changes

SSR Benchmark Configuration

Layer / File(s) Summary
Production environment configuration
benchmarks/ssr/package.json, benchmarks/client-nav/package.json
test:perf scripts now prefix vitest bench with NODE_ENV=production. Dev dependency @codspeed/vitest-plugin bumped to ^5.5.0 in affected benchmark packages.
React: eager module import & bench wiring
benchmarks/ssr/react/speed.bench.ts
Compute appModuleUrl from import.meta.url, top-level dynamic import the SSR module to get the default StartRequestHandler, and run runSsrRequestLoop(handler, ...) directly; remove beforeAll/afterAll and bench setup/teardown.
Solid: eager module import & bench wiring
benchmarks/ssr/solid/speed.bench.ts
Remove suite-hook utilities, compute absolute appModuleUrl, top-level await import(/* @vite-ignore */ appModuleUrl) to obtain default handler, and run runSsrRequestLoop(handler, { seed }) without lifecycle hooks or bench setup/teardown.
Vue: eager module import & bench wiring
benchmarks/ssr/vue/speed.bench.ts
Compute absolute appModuleUrl, top-level dynamic import the SSR module to extract default handler, and invoke runSsrRequestLoop(handler, { seed: benchmarkSeed }) directly; remove deferred init, beforeAll/afterAll, and bench setup/teardown.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested reviewers

  • SeanCassiere

Poem

🐰
I hopped through benches, tidy and spry,
File URLs gleam under import.meta's sky.
Vite ignores, the handlers arrive,
No setup fuss — the loops revive.
A rabbit cheers — the benches thrive.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix(benchmarks): run SSR setup imports natively' directly describes the main change: refactoring SSR benchmarks to use native file URLs for setup imports outside the measured benchmark body, as confirmed by the PR objectives and file summaries.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix-ssr-codspeed-benchmark

Comment @coderabbitai help to get the list of available commands and usage tips.

@nx-cloud
Copy link
Copy Markdown
Contributor

nx-cloud Bot commented Jun 7, 2026

View your CI Pipeline Execution ↗ for commit b57ae87

Command Status Duration Result
nx affected --targets=test:eslint,test:unit,tes... ✅ Succeeded 1m 46s View ↗
nx run-many --target=build --exclude=examples/*... ✅ Succeeded 25s View ↗

💡 Verify your cache is correct by running tasks in a sandbox. Read docs ↗


☁️ Nx Cloud last updated this comment at 2026-06-07 16:03:24 UTC

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 7, 2026

🚀 Changeset Version Preview

1 package(s) bumped directly, 4 bumped as dependents.

🟩 Patch bumps

Package Version Reason
@tanstack/react-router 1.170.15 → 1.170.16 Changeset
@tanstack/react-start 1.168.25 → 1.168.26 Dependent
@tanstack/react-start-client 1.168.13 → 1.168.14 Dependent
@tanstack/react-start-rsc 0.1.24 → 0.1.25 Dependent
@tanstack/react-start-server 1.167.19 → 1.167.20 Dependent

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 7, 2026

Bundle Size Benchmarks

  • Commit: 127243e314f3
  • Measured at: 2026-06-07T16:02:49.904Z
  • Baseline source: history:4a93cffffca3
  • Dashboard: bundle-size history
Scenario Current (gzip) Delta vs baseline Initial gzip Raw Brotli Trend
react-router.minimal 87.25 KiB 0 B (0.00%) 87.11 KiB 273.66 KiB 75.97 KiB ▁▁▁███▃▃▃▃▃
react-router.full 91.08 KiB 0 B (0.00%) 90.94 KiB 286.05 KiB 79.21 KiB ▁▁▁███▇▇▇▇▇
solid-router.minimal 35.49 KiB 0 B (0.00%) 35.36 KiB 105.96 KiB 32.00 KiB ▁▁▁▆▆▆█████
solid-router.full 40.54 KiB 0 B (0.00%) 40.41 KiB 121.17 KiB 36.49 KiB ▁▁▁███▆▆▆▆▆
vue-router.minimal 52.97 KiB 0 B (0.00%) 52.84 KiB 149.97 KiB 47.65 KiB ▁▁▁████████
vue-router.full 58.95 KiB 0 B (0.00%) 58.82 KiB 168.73 KiB 52.81 KiB ▁▁▁███▇▇▇▇▇
react-start.minimal 101.88 KiB 0 B (0.00%) 101.74 KiB 321.97 KiB 88.22 KiB ▁▁▁███▇▇▇▇▇
react-start.deferred-hydration 102.61 KiB 0 B (0.00%) 101.76 KiB 323.35 KiB 88.95 KiB ▁▁▁███▅▅▅▅▅
react-start.full 105.33 KiB 0 B (0.00%) 105.20 KiB 332.40 KiB 91.13 KiB ▁▁▁▅▅▅▅▅███
react-start.rsbuild.minimal 99.60 KiB 0 B (0.00%) 99.43 KiB 316.46 KiB 85.65 KiB ▁▁▁▆▆▆█████
react-start.rsbuild.minimal-iife 100.00 KiB 0 B (0.00%) 99.83 KiB 317.39 KiB 86.08 KiB ▁▁▁▆▆▆█████
react-start.rsbuild.full 102.89 KiB 0 B (0.00%) 102.72 KiB 326.96 KiB 88.50 KiB ▁▁▁▃▃▃▄▄███
solid-start.minimal 49.60 KiB 0 B (0.00%) 49.47 KiB 152.02 KiB 43.81 KiB ▁▁▁▆▆▆█████
solid-start.deferred-hydration 52.86 KiB 0 B (0.00%) 49.53 KiB 160.06 KiB 46.77 KiB ▁▁▁▆▆▆█████
solid-start.full 55.41 KiB 0 B (0.00%) 55.28 KiB 169.07 KiB 48.79 KiB ▁▁▁▄▄▄▂▂███
vue-start.minimal 71.01 KiB 0 B (0.00%) 70.89 KiB 207.11 KiB 62.92 KiB ▁▁▁████████
vue-start.full 75.01 KiB 0 B (0.00%) 74.88 KiB 219.75 KiB 66.39 KiB ▁▁▁▄▄▄▃▃███

Current gzip tracks all emitted client JS chunks. Initial gzip tracks only the entry/import graph. Trend sparkline is historical current gzip ending with this PR measurement; lower is better.

@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new Bot commented Jun 7, 2026

More templates

@tanstack/arktype-adapter

npm i https://pkg.pr.new/@tanstack/arktype-adapter@7579

@tanstack/eslint-plugin-router

npm i https://pkg.pr.new/@tanstack/eslint-plugin-router@7579

@tanstack/eslint-plugin-start

npm i https://pkg.pr.new/@tanstack/eslint-plugin-start@7579

@tanstack/history

npm i https://pkg.pr.new/@tanstack/history@7579

@tanstack/nitro-v2-vite-plugin

npm i https://pkg.pr.new/@tanstack/nitro-v2-vite-plugin@7579

@tanstack/react-router

npm i https://pkg.pr.new/@tanstack/react-router@7579

@tanstack/react-router-devtools

npm i https://pkg.pr.new/@tanstack/react-router-devtools@7579

@tanstack/react-router-ssr-query

npm i https://pkg.pr.new/@tanstack/react-router-ssr-query@7579

@tanstack/react-start

npm i https://pkg.pr.new/@tanstack/react-start@7579

@tanstack/react-start-client

npm i https://pkg.pr.new/@tanstack/react-start-client@7579

@tanstack/react-start-rsc

npm i https://pkg.pr.new/@tanstack/react-start-rsc@7579

@tanstack/react-start-server

npm i https://pkg.pr.new/@tanstack/react-start-server@7579

@tanstack/router-cli

npm i https://pkg.pr.new/@tanstack/router-cli@7579

@tanstack/router-core

npm i https://pkg.pr.new/@tanstack/router-core@7579

@tanstack/router-devtools

npm i https://pkg.pr.new/@tanstack/router-devtools@7579

@tanstack/router-devtools-core

npm i https://pkg.pr.new/@tanstack/router-devtools-core@7579

@tanstack/router-generator

npm i https://pkg.pr.new/@tanstack/router-generator@7579

@tanstack/router-plugin

npm i https://pkg.pr.new/@tanstack/router-plugin@7579

@tanstack/router-ssr-query-core

npm i https://pkg.pr.new/@tanstack/router-ssr-query-core@7579

@tanstack/router-utils

npm i https://pkg.pr.new/@tanstack/router-utils@7579

@tanstack/router-vite-plugin

npm i https://pkg.pr.new/@tanstack/router-vite-plugin@7579

@tanstack/solid-router

npm i https://pkg.pr.new/@tanstack/solid-router@7579

@tanstack/solid-router-devtools

npm i https://pkg.pr.new/@tanstack/solid-router-devtools@7579

@tanstack/solid-router-ssr-query

npm i https://pkg.pr.new/@tanstack/solid-router-ssr-query@7579

@tanstack/solid-start

npm i https://pkg.pr.new/@tanstack/solid-start@7579

@tanstack/solid-start-client

npm i https://pkg.pr.new/@tanstack/solid-start-client@7579

@tanstack/solid-start-server

npm i https://pkg.pr.new/@tanstack/solid-start-server@7579

@tanstack/start-client-core

npm i https://pkg.pr.new/@tanstack/start-client-core@7579

@tanstack/start-fn-stubs

npm i https://pkg.pr.new/@tanstack/start-fn-stubs@7579

@tanstack/start-plugin-core

npm i https://pkg.pr.new/@tanstack/start-plugin-core@7579

@tanstack/start-server-core

npm i https://pkg.pr.new/@tanstack/start-server-core@7579

@tanstack/start-static-server-functions

npm i https://pkg.pr.new/@tanstack/start-static-server-functions@7579

@tanstack/start-storage-context

npm i https://pkg.pr.new/@tanstack/start-storage-context@7579

@tanstack/valibot-adapter

npm i https://pkg.pr.new/@tanstack/valibot-adapter@7579

@tanstack/virtual-file-routes

npm i https://pkg.pr.new/@tanstack/virtual-file-routes@7579

@tanstack/vue-router

npm i https://pkg.pr.new/@tanstack/vue-router@7579

@tanstack/vue-router-devtools

npm i https://pkg.pr.new/@tanstack/vue-router-devtools@7579

@tanstack/vue-router-ssr-query

npm i https://pkg.pr.new/@tanstack/vue-router-ssr-query@7579

@tanstack/vue-start

npm i https://pkg.pr.new/@tanstack/vue-start@7579

@tanstack/vue-start-client

npm i https://pkg.pr.new/@tanstack/vue-start-client@7579

@tanstack/vue-start-server

npm i https://pkg.pr.new/@tanstack/vue-start-server@7579

@tanstack/zod-adapter

npm i https://pkg.pr.new/@tanstack/zod-adapter@7579

commit: b57ae87

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Jun 7, 2026

Merging this PR will improve performance by 82.36%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
❌ 1 (👁 1) regressed benchmark
✅ 3 untouched benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
ssr request loop (react) 327.9 ms 84.2 ms ×3.9
ssr request loop (vue) 481.7 ms 289.6 ms +66.34%
👁 client-side navigation loop (react) 55.8 ms 59.6 ms -6.35%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing fix-ssr-codspeed-benchmark (b57ae87) with main (4a93cff)

Open in CodSpeed

@socket-security
Copy link
Copy Markdown

socket-security Bot commented Jun 7, 2026

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Added@​codspeed/​vitest-plugin@​5.5.0971009896100

View full report

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
benchmarks/ssr/react/speed.bench.ts (1)

1-45: ⚖️ Poor tradeoff

Consider extracting the shared benchmark pattern.

All three SSR benchmark files (React, Solid, Vue) follow an identical structure, differing only in the benchmark seed value and framework label. While this duplication ensures each benchmark remains self-contained, extracting the common lazy-loading, setup/teardown, and benchmarking logic into a shared utility could improve maintainability if this pattern evolves.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@benchmarks/ssr/react/speed.bench.ts` around lines 1 - 45, Extract the
repeated lazy-loading and benchmark wiring into a shared helper (e.g., a factory
like createSsrBenchmark) in bench-utils: move the logic from loadHandler, setup,
teardown, runBenchmark and the import of appModuleUrl into that helper and
parameterize it by module URL and seed (benchmarkSeed) so each framework file
only calls the helper with its specific appModuleUrl and seed and supplies the
framework label to describe/bench; update the current file to import and invoke
that helper, removing duplicated functions and keeping only the small
per-framework constants and the bench() call.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@benchmarks/ssr/react/speed.bench.ts`:
- Around line 1-45: Extract the repeated lazy-loading and benchmark wiring into
a shared helper (e.g., a factory like createSsrBenchmark) in bench-utils: move
the logic from loadHandler, setup, teardown, runBenchmark and the import of
appModuleUrl into that helper and parameterize it by module URL and seed
(benchmarkSeed) so each framework file only calls the helper with its specific
appModuleUrl and seed and supplies the framework label to describe/bench; update
the current file to import and invoke that helper, removing duplicated functions
and keeping only the small per-framework constants and the bench() call.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9779c565-62a4-4dd9-a795-0c2507b177d9

📥 Commits

Reviewing files that changed from the base of the PR and between 7c907ee and 63dbfbe.

📒 Files selected for processing (3)
  • benchmarks/ssr/react/speed.bench.ts
  • benchmarks/ssr/solid/speed.bench.ts
  • benchmarks/ssr/vue/speed.bench.ts

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
benchmarks/ssr/react/speed.bench.ts (1)

8-12: 💤 Low value

Optional: Consider extracting the common pattern across React/Solid/Vue benchmarks.

All three SSR benchmark files (React, Solid, Vue) follow an identical structure: compute appModuleUrl, top-level dynamic import with /* @vite-ignore */, destructure and type-assert the handler, then call runSsrRequestLoop directly in the benchmark body. The only differences are the seed values and benchmark names.

While the current duplication is minimal and benchmark independence may be preferred, extracting a shared factory helper could reduce maintenance overhead if the pattern evolves.

Example shared factory approach
// bench-utils.ts or new file
export function createSsrBenchmark(
  framework: string,
  seed: number,
) {
  const appModuleUrl = new URL(`./dist/server/server.js`, import.meta.url).href
  
  return {
    appModuleUrl,
    seed,
    async loadHandler() {
      const { default: handler } = (await import(
        /* `@vite-ignore` */ appModuleUrl
      )) as { default: StartRequestHandler }
      return handler
    },
  }
}

Then in each benchmark file:

const { seed, loadHandler } = createSsrBenchmark('react', 0xdecafbad)
const handler = await loadHandler()
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@benchmarks/ssr/react/speed.bench.ts` around lines 8 - 12, Extract the
duplicated SSR benchmark pattern into a shared factory (e.g.,
createSsrBenchmark) that returns appModuleUrl, seed, and a loadHandler() which
performs the top-level dynamic import using /* `@vite-ignore` */ and type-asserts
the default as StartRequestHandler; then update each framework benchmark
(React/Solid/Vue) to call createSsrBenchmark(...), await loadHandler() to get
handler, and pass that handler into runSsrRequestLoop, keeping only
framework-specific seed and benchmark name in each file.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@benchmarks/ssr/react/speed.bench.ts`:
- Around line 8-12: Extract the duplicated SSR benchmark pattern into a shared
factory (e.g., createSsrBenchmark) that returns appModuleUrl, seed, and a
loadHandler() which performs the top-level dynamic import using /* `@vite-ignore`
*/ and type-asserts the default as StartRequestHandler; then update each
framework benchmark (React/Solid/Vue) to call createSsrBenchmark(...), await
loadHandler() to get handler, and pass that handler into runSsrRequestLoop,
keeping only framework-specific seed and benchmark name in each file.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d5d96e60-d86e-4ca4-b18c-6a63d5fbcfe0

📥 Commits

Reviewing files that changed from the base of the PR and between 63dbfbe and 6e19939.

📒 Files selected for processing (3)
  • benchmarks/ssr/react/speed.bench.ts
  • benchmarks/ssr/solid/speed.bench.ts
  • benchmarks/ssr/vue/speed.bench.ts

@Sheraff Sheraff requested a review from a team as a code owner June 7, 2026 16:00
@schiller-manuel schiller-manuel merged commit b80781e into main Jun 7, 2026
20 checks passed
@schiller-manuel schiller-manuel deleted the fix-ssr-codspeed-benchmark branch June 7, 2026 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants