Skip to content

[claude] feat(bench): emit v3 JSONL records and dual-write to bench server #7780

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
connortsui20 merged 3 commits into from
May 4, 2026

remove q0 for most groups

5bd762b
Select commit
Loading
Failed to load commit list.
Merged

[claude] feat(bench): emit v3 JSONL records and dual-write to bench server #7780

remove q0 for most groups
5bd762b
Select commit
Loading
Failed to load commit list.
CodSpeed HQ / CodSpeed Performance Analysis failed May 4, 2026 in 0s

Performance Regression: -17.47%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 10 improved benchmarks
❌ 4 regressed benchmarks
✅ 1155 untouched benchmarks
⏩ 138 skipped benchmarks1

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ Simulation take_10k_first_chunk_only 270.8 µs 226 µs +19.84%
⚡ Simulation take_10k_dispersed 284.8 µs 239.8 µs +18.76%
⚡ Simulation patched_take_10k_dispersed 316 µs 285.6 µs +10.64%
⚡ Simulation patched_take_10k_first_chunk_only 302.4 µs 272.1 µs +11.14%
⚡ Simulation patched_take_10k_adversarial 259 µs 228.7 µs +13.23%
⚡ Simulation decompress_rd[f64, (10000, 0.1)] 138.7 µs 122.2 µs +13.48%
❌ Simulation decompress_rd[f32, (100000, 0.01)] 495 µs 582.7 µs -15.06%
❌ Simulation decompress_rd[f64, (100000, 0.01)] 842.6 µs 1,020.7 µs -17.45%
⚡ Simulation decompress_rd[f64, (10000, 0.0)] 138.5 µs 122.2 µs +13.33%
⚡ Simulation decompress_rd[f32, (100000, 0.0)] 583.4 µs 495.8 µs +17.67%
❌ Simulation decompress_rd[f64, (100000, 0.1)] 842.5 µs 1,020.9 µs -17.47%
⚡ Simulation decompress_rd[f64, (10000, 0.01)] 138.6 µs 122.1 µs +13.56%
❌ Simulation decompress_rd[f32, (100000, 0.1)] 495 µs 582.7 µs -15.06%
⚡ Simulation decompress_rd[f32, (10000, 0.0)] 94.5 µs 85.9 µs +10.08%

Comparing claude/benchmarks-v3-emitter-split (5bd762b) with develop (e0a0527)

Open in CodSpeed

Footnotes

  1. 138 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

View more details on CodSpeed HQ