Skip to content

test: faster benchmark ci#6834

Merged
Sheraff merged 4 commits intomainfrom
opencode/hidden-cabin
Mar 6, 2026
Merged

test: faster benchmark ci#6834
Sheraff merged 4 commits intomainfrom
opencode/hidden-cabin

Conversation

@Sheraff
Copy link
Contributor

@Sheraff Sheraff commented Mar 6, 2026

Summary by CodeRabbit

  • Chores
    • Renamed and reorganized the benchmark workflow into a generic, matrix-driven job covering multiple frameworks and benchmark types.
    • Consolidated per-framework/per-benchmark steps into a single matrix-powered benchmarking step that runs combinations in parallel (fail-fast disabled).
    • Updated workflow checkout to a newer action version.
    • Switched build/test dependency handling to explicit project targets instead of a wildcard.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 6, 2026

📝 Walkthrough

Walkthrough

Converted the single-framework CI workflow into a matrix-based "Benchmarks" workflow (framework: react/solid/vue × benchmark: client-nav/ssr), consolidated per-matrix CodSpeed steps into one matrix-driven step, updated actions/checkout to v6.0.1, and replaced ^build with explicit NX targets in benchmarks/ssr/package.json.

Changes

Cohort / File(s) Summary
CI - Benchmarks workflow
.github/workflows/client-nav-benchmarks.yml
Renamed workflow to "Benchmarks"; changed job to a matrix over framework (react, solid, vue) and benchmark (client-nav, ssr); replaced per-framework steps with a single matrix-driven CodSpeed run step using matrix.framework/matrix.benchmark; updated actions/checkout to v6.0.1; fail-fast: false.
SSR benchmark package config
benchmarks/ssr/package.json
Replaced wildcard ^build dependency with explicit NX targets build:react, build:solid, build:vue; removed implicit wildcard pre-build dependency in test:perf, making build prerequisites explicit.

Sequence Diagram(s)

sequenceDiagram
  participant GH as GitHub Actions
  participant Checkout as actions/checkout@v6.0.1
  participant Runner as CI Runner
  participant CodSpeed as CodSpeed
  participant NX as pnpm nx

  GH->>Runner: matrix job (framework, benchmark)
  Runner->>Checkout: checkout repo
  Runner->>Runner: setup Node, pnpm, tools
  Runner->>CodSpeed: Run CodSpeed (WITH_INSTRUMENTATION=1)
  CodSpeed->>NX: pnpm nx run `@benchmarks/`${matrix.benchmark}:test:perf:${matrix.framework}
  NX->>Runner: build/tests (build:react|solid|vue as needed)
  NX->>CodSpeed: perf results
  CodSpeed->>GH: upload/report results
Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 I hopped through YAML fields with glee,
Matrix legs dancing — a trifecta of three.
Wildcards replaced by targets so neat,
CodSpeed hums as benchmarks meet.
🥕

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'test: faster benchmark ci' directly relates to the main changes in the PR, which optimize the benchmark CI workflow by introducing matrix-based parallel execution and consolidating configuration.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch opencode/hidden-cabin

Comment @coderabbitai help to get the list of available commands and usage tips.

@nx-cloud
Copy link

nx-cloud bot commented Mar 6, 2026

View your CI Pipeline Execution ↗ for commit 01dda17

Command Status Duration Result
nx affected --targets=test:eslint,test:unit,tes... ✅ Succeeded 48s View ↗
nx run-many --target=build --exclude=examples/*... ✅ Succeeded 1s View ↗

☁️ Nx Cloud last updated this comment at 2026-03-06 14:26:48 UTC

@pkg-pr-new
Copy link

pkg-pr-new bot commented Mar 6, 2026

More templates

@tanstack/arktype-adapter

npm i https://pkg.pr.new/@tanstack/arktype-adapter@6834

@tanstack/eslint-plugin-router

npm i https://pkg.pr.new/@tanstack/eslint-plugin-router@6834

@tanstack/history

npm i https://pkg.pr.new/@tanstack/history@6834

@tanstack/nitro-v2-vite-plugin

npm i https://pkg.pr.new/@tanstack/nitro-v2-vite-plugin@6834

@tanstack/react-router

npm i https://pkg.pr.new/@tanstack/react-router@6834

@tanstack/react-router-devtools

npm i https://pkg.pr.new/@tanstack/react-router-devtools@6834

@tanstack/react-router-ssr-query

npm i https://pkg.pr.new/@tanstack/react-router-ssr-query@6834

@tanstack/react-start

npm i https://pkg.pr.new/@tanstack/react-start@6834

@tanstack/react-start-client

npm i https://pkg.pr.new/@tanstack/react-start-client@6834

@tanstack/react-start-server

npm i https://pkg.pr.new/@tanstack/react-start-server@6834

@tanstack/router-cli

npm i https://pkg.pr.new/@tanstack/router-cli@6834

@tanstack/router-core

npm i https://pkg.pr.new/@tanstack/router-core@6834

@tanstack/router-devtools

npm i https://pkg.pr.new/@tanstack/router-devtools@6834

@tanstack/router-devtools-core

npm i https://pkg.pr.new/@tanstack/router-devtools-core@6834

@tanstack/router-generator

npm i https://pkg.pr.new/@tanstack/router-generator@6834

@tanstack/router-plugin

npm i https://pkg.pr.new/@tanstack/router-plugin@6834

@tanstack/router-ssr-query-core

npm i https://pkg.pr.new/@tanstack/router-ssr-query-core@6834

@tanstack/router-utils

npm i https://pkg.pr.new/@tanstack/router-utils@6834

@tanstack/router-vite-plugin

npm i https://pkg.pr.new/@tanstack/router-vite-plugin@6834

@tanstack/solid-router

npm i https://pkg.pr.new/@tanstack/solid-router@6834

@tanstack/solid-router-devtools

npm i https://pkg.pr.new/@tanstack/solid-router-devtools@6834

@tanstack/solid-router-ssr-query

npm i https://pkg.pr.new/@tanstack/solid-router-ssr-query@6834

@tanstack/solid-start

npm i https://pkg.pr.new/@tanstack/solid-start@6834

@tanstack/solid-start-client

npm i https://pkg.pr.new/@tanstack/solid-start-client@6834

@tanstack/solid-start-server

npm i https://pkg.pr.new/@tanstack/solid-start-server@6834

@tanstack/start-client-core

npm i https://pkg.pr.new/@tanstack/start-client-core@6834

@tanstack/start-fn-stubs

npm i https://pkg.pr.new/@tanstack/start-fn-stubs@6834

@tanstack/start-plugin-core

npm i https://pkg.pr.new/@tanstack/start-plugin-core@6834

@tanstack/start-server-core

npm i https://pkg.pr.new/@tanstack/start-server-core@6834

@tanstack/start-static-server-functions

npm i https://pkg.pr.new/@tanstack/start-static-server-functions@6834

@tanstack/start-storage-context

npm i https://pkg.pr.new/@tanstack/start-storage-context@6834

@tanstack/valibot-adapter

npm i https://pkg.pr.new/@tanstack/valibot-adapter@6834

@tanstack/virtual-file-routes

npm i https://pkg.pr.new/@tanstack/virtual-file-routes@6834

@tanstack/vue-router

npm i https://pkg.pr.new/@tanstack/vue-router@6834

@tanstack/vue-router-devtools

npm i https://pkg.pr.new/@tanstack/vue-router-devtools@6834

@tanstack/vue-router-ssr-query

npm i https://pkg.pr.new/@tanstack/vue-router-ssr-query@6834

@tanstack/vue-start

npm i https://pkg.pr.new/@tanstack/vue-start@6834

@tanstack/vue-start-client

npm i https://pkg.pr.new/@tanstack/vue-start-client@6834

@tanstack/vue-start-server

npm i https://pkg.pr.new/@tanstack/vue-start-server@6834

@tanstack/zod-adapter

npm i https://pkg.pr.new/@tanstack/zod-adapter@6834

commit: afcbb18

@github-actions
Copy link
Contributor

github-actions bot commented Mar 6, 2026

Bundle Size Benchmarks

  • Commit: 023fdfd15270
  • Measured at: 2026-03-06T14:27:06.109Z
  • Baseline source: history:023fdfd15270
  • Dashboard: bundle-size history
Scenario Current (gzip) Delta vs baseline Raw Brotli Trend
react-router.minimal 86.84 KiB 0 B (0.00%) 273.33 KiB 75.44 KiB ▁▁▁▁▁▁▁████
react-router.full 89.88 KiB 0 B (0.00%) 283.67 KiB 78.18 KiB ▁▁▁▁▁▁▁████
solid-router.minimal 36.17 KiB 0 B (0.00%) 108.44 KiB 32.49 KiB ▁▁▁▁▁▁▁████
solid-router.full 40.50 KiB 0 B (0.00%) 121.49 KiB 36.40 KiB ▁▁▁▁▁▁▁████
vue-router.minimal 52.03 KiB 0 B (0.00%) 148.42 KiB 46.71 KiB ▁▁▁▁▁▁▁████
vue-router.full 56.85 KiB 0 B (0.00%) 164.01 KiB 51.04 KiB ▁▁▁▁▁▁▁████
react-start.minimal 99.40 KiB 0 B (0.00%) 312.48 KiB 85.98 KiB ▁▁▁▁▁▁▁████
react-start.full 102.78 KiB 0 B (0.00%) 322.29 KiB 88.82 KiB ▁▁▁▁▁▁▁████
solid-start.minimal 48.49 KiB 0 B (0.00%) 146.03 KiB 42.85 KiB ▁▁▁▁▁▁▁████
solid-start.full 53.96 KiB 0 B (0.00%) 161.98 KiB 47.51 KiB ▁▁▁▁▁▁▁████

Trend sparkline is historical gzip bytes ending with this PR measurement; lower is better.

@codspeed-hq
Copy link

codspeed-hq bot commented Mar 6, 2026

Merging this PR will not alter performance

✅ 6 untouched benchmarks


Comparing opencode/hidden-cabin (01dda17) with main (023fdfd)

Open in CodSpeed

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
.github/workflows/client-nav-benchmarks.yml (1)

21-33: Optional: add a job timeout to avoid stuck benchmark runners.

As this matrix expands, a timeout helps keep CI capacity predictable when a benchmark hangs.

⏱️ Suggested hardening
   benchmarks:
     name: Run ${{ matrix.benchmark }}:${{ matrix.framework }} CodSpeed benchmark
+    timeout-minutes: 20
     strategy:
       fail-fast: false
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/client-nav-benchmarks.yml around lines 21 - 33, The
benchmarks job ("benchmarks") currently has no timeout; add a job-level timeout
to avoid stuck runners by inserting a timeout-minutes key (e.g.,
timeout-minutes: 30) under the job definition for "benchmarks" so the matrix
runs (framework/benchmark) are forcibly cancelled after the configured duration;
update the workflow YAML near the "benchmarks:" block that contains "strategy:",
"matrix:", and "runs-on:" to include the timeout-minutes setting.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In @.github/workflows/client-nav-benchmarks.yml:
- Around line 21-33: The benchmarks job ("benchmarks") currently has no timeout;
add a job-level timeout to avoid stuck runners by inserting a timeout-minutes
key (e.g., timeout-minutes: 30) under the job definition for "benchmarks" so the
matrix runs (framework/benchmark) are forcibly cancelled after the configured
duration; update the workflow YAML near the "benchmarks:" block that contains
"strategy:", "matrix:", and "runs-on:" to include the timeout-minutes setting.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 91ca100c-12b2-47ac-9054-d5979d365501

📥 Commits

Reviewing files that changed from the base of the PR and between d92d1ab and 01dda17.

📒 Files selected for processing (1)
  • .github/workflows/client-nav-benchmarks.yml

@Sheraff Sheraff merged commit b46b2ee into main Mar 6, 2026
15 checks passed
@Sheraff Sheraff deleted the opencode/hidden-cabin branch March 6, 2026 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant