Fix #503 pricing coverage and unpriced hotspot ranking - #510
Conversation
|
Warning Review limit reached
Next review available in: 34 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (16)
📝 WalkthroughWalkthroughThe SDK now detects unknown model pricing, creates token-ranked unpriced findings, and exposes pricing status. Hotspot queries and CLI rendering support the new finding kind. Pricing flattening preserves primary-provider entries. Node types and changelogs document the output. ChangesUnpriced hotspot reporting
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant HotspotsQuery
participant FindingAnalyzer
participant HotspotRenderer
HotspotsQuery->>FindingAnalyzer: request unpriced-usage findings
FindingAnalyzer->>FindingAnalyzer: aggregate unknown-price turns by session and tokens
FindingAnalyzer->>HotspotsQuery: return token-ranked findings
HotspotsQuery->>HotspotRenderer: pass pricing status and savings
HotspotRenderer->>HotspotRenderer: render unpriced label
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
💡 Codex Review Preserve Google’s first-party tariffs too
When the refreshed snapshot contains a Google model under both https://github.com/AgentWorkforce/burn/blob/7dd61144c652dd95f8f6c1d8702cff6a6b3d028a/crates/relayburn-sdk/data/models.dev.json#L1 For turns beyond the context threshold, the newly vendored  Propagate unpriced status into detector findings
When an unknown-model turn also forms a retry, failure, edit, or skill finding, the detector cost path uses ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (2)
crates/relayburn-sdk/src/analyze/pricing.rs (1)
302-322: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd a reverse-order case to the primary-precedence test.
This test only covers the case where
openaiappears beforeresellerin the source JSON. Theflattenimplementation also correctly protects primary pricing when the reseller entry appears first in file order (the reseller entry gets overwritten later when the primary provider is processed). Add a second test case withresellerlisted beforeopenaito lock in that direction too, sincemodels.dev.jsonprovider ordering is not guaranteed and the code comment at Line 87 explicitly notes "their order is not a pricing-precedence contract."🧪 Suggested additional test
+ #[test] + fn flatten_prefers_primary_vendor_when_reseller_listed_first() { + let raw = r#"{ + "reseller": { + "models": { + "gpt-example": { "cost": { "input": 1.5, "output": 12, "reasoning": 12 } } + } + }, + "openai": { + "models": { + "gpt-example": { "cost": { "input": 5, "output": 30 } } + } + } + }"#; + let table = parse_pricing(raw).unwrap(); + let cost = table.get("gpt-example").unwrap(); + assert_eq!(cost.input, 5.0); + assert_eq!(cost.output, 30.0); + assert_eq!(cost.reasoning_mode, ReasoningMode::SameAsOutput); + }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/relayburn-sdk/src/analyze/pricing.rs` around lines 302 - 322, Add a reverse-order case alongside flatten_prefers_primary_vendor_over_later_reseller_copy, placing the reseller provider before openai in the raw JSON while keeping the duplicate model and expected primary pricing assertions. Verify input, output, and reasoning_mode still come from the openai entry.crates/relayburn-sdk/src/query_verbs/tests.rs (1)
1056-1112: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winExercise the ranking path in this test.
This query requests only
unpriced-usageand creates one unpriced session. The assertions verify metadata and serialization, but they cannot detect token-order regressions or verify that unpriced findings precede priced findings. Add a second unpriced session with a smaller token total and include a priced finding in the sorted result set. Assert the expected order.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/relayburn-sdk/src/query_verbs/tests.rs` around lines 1056 - 1112, The test hotspots_findings_surface_unpriced_usage_with_token_rank currently exercises only one unpriced finding, so it cannot validate ranking. Add a second unpriced session with a lower total token count and ensure the fixture also produces a priced finding by broadening the query beyond only unpriced-usage; then assert the sorted findings order, with the higher-token unpriced finding first and unpriced findings preceding the priced finding, while retaining the existing metadata and serialization checks.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@crates/relayburn-sdk/src/analyze/pricing.rs`:
- Around line 302-322: Add a reverse-order case alongside
flatten_prefers_primary_vendor_over_later_reseller_copy, placing the reseller
provider before openai in the raw JSON while keeping the duplicate model and
expected primary pricing assertions. Verify input, output, and reasoning_mode
still come from the openai entry.
In `@crates/relayburn-sdk/src/query_verbs/tests.rs`:
- Around line 1056-1112: The test
hotspots_findings_surface_unpriced_usage_with_token_rank currently exercises
only one unpriced finding, so it cannot validate ranking. Add a second unpriced
session with a lower total token count and ensure the fixture also produces a
priced finding by broadening the query beyond only unpriced-usage; then assert
the sorted findings order, with the higher-token unpriced finding first and
unpriced findings preceding the priced finding, while retaining the existing
metadata and serialization checks.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: d6cd03e7-50f6-4a16-9165-995f3b3330e7
📒 Files selected for processing (17)
CHANGELOG.mdcrates/relayburn-cli/src/commands/hotspots/human.rscrates/relayburn-cli/src/commands/hotspots/mod.rscrates/relayburn-sdk/data/models.dev.jsoncrates/relayburn-sdk/src/analyze.rscrates/relayburn-sdk/src/analyze/findings.rscrates/relayburn-sdk/src/analyze/ghost_surface.rscrates/relayburn-sdk/src/analyze/pricing.rscrates/relayburn-sdk/src/analyze/tool_output_bloat.rscrates/relayburn-sdk/src/lib.rscrates/relayburn-sdk/src/query_verbs/hotspots.rscrates/relayburn-sdk/src/query_verbs/mod.rscrates/relayburn-sdk/src/query_verbs/tests.rspackages/mcp/CHANGELOG.mdpackages/relayburn/CHANGELOG.mdpackages/sdk-node/CHANGELOG.mdpackages/sdk-node/src/index.d.ts
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 17 files
Not reviewed (too large): crates/relayburn-sdk/data/models.dev.json (~2 lines) - if these are generated or fixture files, add them to ignored paths to exclude them from future reviews.
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
|
Validated all three Codex findings and fixed them in aa0ad71:
cargo fmt --all -- --check and cargo test --workspace --quiet both pass (838 passed, 5 ignored in the SDK suite). |
There was a problem hiding this comment.
All reported issues were addressed across 16 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
1 participant
Fixes #503
The vendored models.dev snapshot now includes pricing for claude-opus-5, claude-sonnet-5, gpt-5.6-sol, gpt-5.6-luna, and gpt-5.6-terra. The snapshot comes from the sanctioned models.dev API refresh path.
Pricing flattening gives Anthropic, OpenAI, Google, Google Vertex, and xAI first-party tariffs precedence over duplicate reseller entries, preventing provider file order from silently replacing canonical rates or reasoning semantics. Context-based pricing tiers are preserved and applied when a turn exceeds the provider threshold, including the higher long-context GPT-5.6 rates.
Hotspot findings now emit one explicit unpriced-usage finding per affected session. Unpriced findings carry token volume, sort ahead of priced findings with unknown-price sessions ordered by tokens, render as unpriced in human output, and expose pricingStatus in JSON and Node types without a fake USD estimate. Detector findings from sessions with any unpriced turns also drop partial or zero USD estimates and inherit the visible unpriced status.
Tests cover all five current-generation model IDs, first-party provider precedence, long-context tier selection, token-ranked unpriced findings, detector propagation, JSON status, and the end-to-end hotspot query path.
🤖 Generated with Claude Code.