chore(bench): add --pad-entries to cache_hit_rate for threshold sweeps#33
Conversation
Lets the verify-wiring pad size be tuned independently of the default 160 entries (~8.4K tokens). Used 2026-05-15 to binary-search Claude Haiku 4.5's empirical cache minimum: 4112 tok = miss, 4161 tok = hit → threshold ≈ 4096 tokens (matches Sonnet 4.6's 2K boundary, both clean powers of 2). Documented 1024-token minimum is wrong for Claude 4 family.
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Covers 33 commits since v0.6.1: - TD-194 Council Pilot full merge (#20) + 5 post-merge fix-ups (#22-#26) - TD-189 steps 1-4: per-task cache_hit_rate plumbing (#27-#30) - TD-192: fold OutputRequirementClassifier into FractalBypassClassifier (#31) - Round 22 live latency regression — fractal-entry 2 → 1 LLM calls (#32) - Haiku 4.5 cache threshold pinned at ~4096 tokens via --pad-entries (#33) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Summary
--pad-entries Nflag tobenchmarks/cache_hit_rate.pyso the--verify-wiringpad size can be tuned independently of the default 160 entries (~8.4K tokens, ~50 tok/entry).Finding (TD-193 follow-up — Haiku 4.5 threshold pinned)
Empirical Claude 4 cache minimums (this account):
Both clean powers of 2; documented 1024-token minimum is wrong for both.
Test plan
ruff check benchmarks/cache_hit_rate.py— clean.claude-haiku-4-5-20251001); each--calls 3 --verify-wiring --pad-entries N. Results above.🤖 Generated with Claude Code