Grok weekly label + local session token analytics - #137
Conversation
Grok puts the shared weekly pool in the primary slot with ~7d window minutes. Bridge maps weekly-cadence primaries to metadata.weekly_label, which was wrongly set to Extra credits (intended only for optional prepaid). Label both session and weekly as Weekly so the taskbar and popout match Claude/Codex weekly wording.
Scan ~/.grok/sessions updates.jsonl turn_completed usage for token totals, cache reads, reasoning tokens, reasoning-effort, and project rollups. Wire Grok into the shared cost report path so Charts shows Claude/Codex-style local usage cards (unpriced SuperGrok pool, no fabricated dollars).
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
ceiling | 76042ae | Commit Preview URL Branch Preview URL |
Jul 24 2026, 04:07 AM |
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 48 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughChangesGrok local usage integration
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant GrokSessionLogs
participant RustCostScanner
participant ChartCommands
participant ChartsSection
GrokSessionLogs->>RustCostScanner: provide updates.jsonl and summary.json
RustCostScanner->>RustCostScanner: parse, deduplicate, and aggregate usage
RustCostScanner-->>ChartCommands: return CostUsageReport
ChartCommands-->>ChartsSection: return LocalTokenBreakdown
ChartsSection->>ChartsSection: render token and reasoning buckets
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Address CI Rust shared failure: format imports and helpers, drop unnecessary i64 cast, collapse nested dedup if.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (1)
apps/desktop-tauri/src-tauri/src/commands/chart.rs (1)
1343-1359: 🚀 Performance & Scalability | 🔵 Trivial | ⚖️ Poor tradeoffGrok scans are not cancellable.
Unlike codex/claude, the "grok" arm discards
cancelentirely and runsscan_grok_reportto completion. This is explicitly called out in the comment, so likely a deliberate, scoped tradeoff for this PR — flagging in case a large~/.grok/sessionstree makes this noticeably slow to abandon when the panel closes mid-scan.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/desktop-tauri/src-tauri/src/commands/chart.rs` around lines 1343 - 1359, Update the “grok” branch in scan_local_cost to use a cancellation-aware Grok scanning path, such as scan_grok_report if it accepts cancel, and propagate the cancel token instead of discarding it. Preserve the existing CostSummary result mapping and provider behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@rust/src/cost_scanner.rs`:
- Around line 2386-2419: Serialize the process-wide GROK_HOME mutation in
grok_report_rolls_up_tokens_cache_effort_and_project by acquiring a dedicated
grok_home_lock(), following the existing codex_home_lock() convention in this
file. Ensure the lock remains held through setting GROK_HOME, calling
scan_grok_report, and restoring the previous environment value.
- Around line 1545-1558: Update scan_grok_report to derive the Grok sessions
directory from the scoped home on the provided CostScanner instead of calling
grok_sessions_dir(None). Pass the scanner’s configured home through the existing
directory-resolution path, matching the Codex and Claude scan implementations
while preserving the current missing-directory early returns.
- Around line 1501-1543: Update add_grok_record_to_summary so reasoning tokens
are added to processed_tokens alongside output tokens in the overall, by-model,
by-effort, and by-project token totals, matching the existing
NormalizedTokens::processed behavior. Adjust the Grok chart test’s expected
processed-token total accordingly.
In `@rust/src/grok_costs.rs`:
- Around line 95-106: Update the cwd-to-project conversion in the parser to call
the shared project_from_cwd helper instead of constructing a Path and extracting
file_name locally. Preserve the existing optional cwd handling and trimming, and
reuse the helper’s support for Windows, UNC, and POSIX paths.
---
Nitpick comments:
In `@apps/desktop-tauri/src-tauri/src/commands/chart.rs`:
- Around line 1343-1359: Update the “grok” branch in scan_local_cost to use a
cancellation-aware Grok scanning path, such as scan_grok_report if it accepts
cancel, and propagate the cancel token instead of discarding it. Preserve the
existing CostSummary result mapping and provider behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 763219e7-76ba-4324-9c03-b08aefbc3610
📒 Files selected for processing (13)
apps/desktop-tauri/src-tauri/src/commands/chart.rsapps/desktop-tauri/src/i18n/keys.tsapps/desktop-tauri/src/lib/providerCharts.tsapps/desktop-tauri/src/surfaces/ChartsPanel.tsxapps/desktop-tauri/src/surfaces/settings/providers/sections/charts/ChartsSection.tsxapps/desktop-tauri/src/types/bridge.tsdocs/DATA_SOURCES.mdrust/src/cost_scanner.rsrust/src/grok_costs.rsrust/src/lib.rsrust/src/locale.rsrust/src/locale/en-US.ftlrust/src/providers/grok/mod.rs
| fn add_grok_record_to_summary(summary: &mut CostSummary, record: &GrokUsageRecord) { | ||
| // No public SuperGrok API rate card — keep dollars unset, still track tokens. | ||
| summary.unknown_models.insert(record.model.clone()); | ||
| summary.input_tokens += record.input; | ||
| summary.output_tokens += record.output; | ||
| summary.cached_tokens += record.cache_read; | ||
| summary.cache_read_tokens += record.cache_read; | ||
| summary.reasoning_tokens += record.reasoning; | ||
|
|
||
| let model_tokens = summary | ||
| .by_model_tokens | ||
| .entry(record.model.clone()) | ||
| .or_default(); | ||
| model_tokens.input_tokens += record.input; | ||
| model_tokens.output_tokens += record.output; | ||
| model_tokens.cached_tokens += record.cache_read; | ||
| model_tokens.cache_read_tokens += record.cache_read; | ||
| model_tokens.calls += 1; | ||
|
|
||
| let effort = match record | ||
| .effort | ||
| .as_deref() | ||
| .map(str::trim) | ||
| .filter(|e| !e.is_empty()) | ||
| { | ||
| Some(effort) => effort.to_ascii_lowercase(), | ||
| None => "unknown".to_string(), | ||
| }; | ||
| let effort_tokens = summary.by_effort_tokens.entry(effort).or_default(); | ||
| effort_tokens.input_tokens += record.input; | ||
| effort_tokens.output_tokens += record.output; | ||
| effort_tokens.cached_tokens += record.cache_read; | ||
| effort_tokens.cache_read_tokens += record.cache_read; | ||
| effort_tokens.calls += 1; | ||
|
|
||
| let project = project_bucket(record.project.as_deref()); | ||
| let project_tokens = summary.by_project_tokens.entry(project).or_default(); | ||
| project_tokens.input_tokens += record.input; | ||
| project_tokens.output_tokens += record.output; | ||
| project_tokens.cached_tokens += record.cache_read; | ||
| project_tokens.cache_read_tokens += record.cache_read; | ||
| project_tokens.calls += 1; | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== cost_scanner references ==\n'
rg -n "reasoning_tokens|outputTokens|processed_tokens|token_breakdown|grok_token_breakdown" rust/src apps/desktop-tauri/src-tauri/src/commands/chart.rs
printf '\n== relevant snippets (cost_scanner) ==\n'
sed -n '1,120p' rust/src/cost_scanner.rs
printf '\n---\n'
sed -n '1480,1565p' rust/src/cost_scanner.rs
printf '\n== relevant snippets (chart) ==\n'
sed -n '1340,1395p' apps/desktop-tauri/src-tauri/src/commands/chart.rs
printf '\n---\n'
sed -n '2228,2285p' apps/desktop-tauri/src-tauri/src/commands/chart.rsRepository: tsouth89/ceiling
Length of output: 19846
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== external docs search target hints ==\n'
python3 - <<'PY'
print("Need xAI/Grok docs on usage.completion_tokens_details.reasoning_tokens and whether completion_tokens includes reasoning.")
PYRepository: tsouth89/ceiling
Length of output: 316
🌐 Web query:
xAI Grok usage completion_tokens_details.reasoning_tokens completion_tokens includes reasoning tokens documentation
💡 Result:
In the xAI Grok API, reasoning tokens are reported in the usage.completion_tokens_details.reasoning_tokens field [1][2]. Critically, xAI's implementation differs from some other providers: completion_tokens represents only the final text output of the model and does not include the reasoning tokens [3][4]. To determine the total number of completion tokens generated, you must manually sum the completion_tokens value and the reasoning_tokens value [4]. The API structure is defined as follows: - usage.completion_tokens: The number of tokens in the final generated text output [3][2]. - usage.completion_tokens_details.reasoning_tokens: The number of tokens generated by the model during its internal reasoning process [1][3][5]. - usage.total_tokens: Typically represents the sum of all tokens (prompt + completion + reasoning), though users should verify their specific implementation's handling of the sum given the distinction between completion and reasoning tokens [1][2][6]. Official documentation and API responses clarify that these fields are distinct to account for the model's internal "thinking" process separately from the final response [1][5][4].
Citations:
- 1: https://docs.x.ai/developers/rest-api-reference/inference/chat
- 2: https://docs.x.ai/developers/rest-api-reference/inference/legacy
- 3: https://docs.x.ai/developers/tools/tool-usage-details
- 4: https://dev.to/maximsaplin/grok-3-api-reasoning-tokens-are-counted-differently-197
- 5: https://docs.x.ai/developers/model-capabilities/text/reasoning
- 6: https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/partner-models/grok/capabilities/reasoning
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n "reasoningTokens|reasoning_tokens|outputTokens|updates\.jsonl|processed_tokens" rust apps docs .github --glob '!**/target/**'Repository: tsouth89/ceiling
Length of output: 6787
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n "fn processed\(|struct LocalTokenBreakdown|normalized_tokens|reasoning_tokens" apps/desktop-tauri/src-tauri/src/commands/chart.rs rust/src -g '!**/target/**'
sed -n '150,230p' apps/desktop-tauri/src-tauri/src/commands/chart.rs
sed -n '230,330p' apps/desktop-tauri/src-tauri/src/commands/chart.rsRepository: tsouth89/ceiling
Length of output: 8904
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
print("Checking how processed tokens are defined for LocalTokenBreakdown and whether Grok normalization excludes reasoning.")
PYRepository: tsouth89/ceiling
Length of output: 271
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '120,210p' rust/src/cost_scanner.rs
printf '\n---\n'
sed -n '180,230p' rust/src/grok_costs.rs
printf '\n---\n'
sed -n '300,340p' rust/src/grok_costs.rsRepository: tsouth89/ceiling
Length of output: 6119
Include Grok reasoning tokens in processed totals
xAI reports reasoningTokens separately from outputTokens, so NormalizedTokens::processed() undercounts Grok usage by that amount. Fold reasoning into processed_tokens here and update the Grok chart test to match.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@rust/src/cost_scanner.rs` around lines 1501 - 1543, Update
add_grok_record_to_summary so reasoning tokens are added to processed_tokens
alongside output tokens in the overall, by-model, by-effort, and by-project
token totals, matching the existing NormalizedTokens::processed behavior. Adjust
the Grok chart test’s expected processed-token total accordingly.
| fn scan_grok_report( | ||
| scanner: &CostScanner, | ||
| days: u32, | ||
| windows: &[CurrentUsageWindow], | ||
| ) -> CostUsageReport { | ||
| let _ = scanner; | ||
| let mut daily = empty_daily_summaries(days); | ||
| let mut current_windows = empty_current_window_summaries(windows); | ||
| let Some(sessions_root) = grok_sessions_dir(None) else { | ||
| return finish_report(daily, days, None, (0, 0, 0), None, current_windows); | ||
| }; | ||
| if !sessions_root.exists() { | ||
| return finish_report(daily, days, None, (0, 0, 0), None, current_windows); | ||
| } |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate Grok-related symbols and nearby tests.
rg -n --hidden -S "fn scan_grok_report|grok_sessions_dir|grok_report_rolls_up_tokens_cache_effort_and_project|codex_home_lock|set_var\\(|remove_var\\(|reasoning_tokens|add_grok_record_to_summary|get_cost_usage_report_scoped|scoped_to\\(" rust/src/cost_scanner.rs
echo "--- scan_grok_report context ---"
sed -n '1520,1595p' rust/src/cost_scanner.rs
echo "--- get_cost_usage_report_scoped context ---"
sed -n '1120,1185p' rust/src/cost_scanner.rs
echo "--- Grok report test context ---"
sed -n '2360,2465p' rust/src/cost_scanner.rs
echo "--- grok_sessions_dir definition ---"
rg -n -A 20 -B 10 "fn grok_sessions_dir" rust/src/cost_scanner.rsRepository: tsouth89/ceiling
Length of output: 11204
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect broader surrounding code that affects token accounting and scope dispatch.
sed -n '1,120p' rust/src/cost_scanner.rs
echo "---"
sed -n '1480,1548p' rust/src/cost_scanner.rs
echo "---"
sed -n '1595,1665p' rust/src/cost_scanner.rsRepository: tsouth89/ceiling
Length of output: 10035
scan_grok_report should use the scoped home.
grok_sessions_dir(None) bypasses the CostScanner passed in here, so a scoped scan still reads the global GROK_HOME and collapses every Grok account into one report. Thread the scanner’s home through this path like the Codex/Claude scans do.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@rust/src/cost_scanner.rs` around lines 1545 - 1558, Update scan_grok_report
to derive the Grok sessions directory from the scoped home on the provided
CostScanner instead of calling grok_sessions_dir(None). Pass the scanner’s
configured home through the existing directory-resolution path, matching the
Codex and Claude scan implementations while preserving the current
missing-directory early returns.
| #[test] | ||
| fn grok_report_rolls_up_tokens_cache_effort_and_project() { | ||
| let home = tempfile::tempdir().unwrap(); | ||
| let session = home | ||
| .path() | ||
| .join("sessions") | ||
| .join("proj") | ||
| .join("019f-session"); | ||
| std::fs::create_dir_all(&session).unwrap(); | ||
| let now = Utc::now(); | ||
| let ts = now.timestamp() as f64; | ||
| let ms = now.timestamp_millis(); | ||
| let updates = format!( | ||
| r#"{{"timestamp":{ts},"method":"_x.ai/session/update","params":{{"sessionId":"s1","_meta":{{"eventId":"e1","agentTimestampMs":{ms}}},"update":{{"sessionUpdate":"turn_completed","prompt_id":"p1","usage":{{"inputTokens":1000,"outputTokens":100,"cachedReadTokens":800,"reasoningTokens":40,"modelUsage":{{"grok-4.5-build":{{"inputTokens":1000,"outputTokens":100,"cachedReadTokens":800,"reasoningTokens":40}}}}}}}}}}}}"# | ||
| ); | ||
| std::fs::write(session.join("updates.jsonl"), updates).unwrap(); | ||
| std::fs::write( | ||
| session.join("summary.json"), | ||
| r#"{"info":{"cwd":"C:\\projects\\personal\\ceiling"},"reasoning_effort":"high","current_model_id":"grok-4.5"}"#, | ||
| ) | ||
| .unwrap(); | ||
|
|
||
| // SAFETY: test-only env override; restored after the scan. | ||
| let prev = std::env::var_os("GROK_HOME"); | ||
| // SAFETY: single-threaded test isolation for GROK_HOME. | ||
| unsafe { | ||
| std::env::set_var("GROK_HOME", home.path()); | ||
| } | ||
| let report = scan_grok_report(&CostScanner::new(7), 7, &[]); | ||
| match prev { | ||
| Some(value) => unsafe { std::env::set_var("GROK_HOME", value) }, | ||
| None => unsafe { std::env::remove_var("GROK_HOME") }, | ||
| } | ||
|
|
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Guard the GROK_HOME env mutation like the existing CODEX_HOME pattern in this file.
This test mutates the process-wide GROK_HOME env var via unsafe set_var/remove_var with no serialization. codex_report_counts_archived_rollouts_exactly_once in this same file already hit this exact hazard for CODEX_HOME and added codex_home_lock() specifically to serialize access across concurrently-run tests. Since cargo test runs tests in parallel by default, any future test that reads GROK_HOME concurrently (directly or via grok_home()) would be flaky. Add an equivalent lock for GROK_HOME now, following the established convention, rather than waiting for a flaky-test bug report.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@rust/src/cost_scanner.rs` around lines 2386 - 2419, Serialize the
process-wide GROK_HOME mutation in
grok_report_rolls_up_tokens_cache_effort_and_project by acquiring a dedicated
grok_home_lock(), following the existing codex_home_lock() convention in this
file. Ensure the lock remains held through setting GROK_HOME, calling
scan_grok_report, and restoring the previous environment value.
| let cwd = value | ||
| .pointer("/info/cwd") | ||
| .and_then(Value::as_str) | ||
| .map(str::trim) | ||
| .filter(|s| !s.is_empty()); | ||
| let project = cwd.map(|path| { | ||
| Path::new(path) | ||
| .file_name() | ||
| .and_then(|n| n.to_str()) | ||
| .unwrap_or(path) | ||
| .to_string() | ||
| }); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
set -euo pipefail
printf '\n== file list ==\n'
git ls-files 'rust/src/*.rs' | sed -n '1,120p'
printf '\n== grok_costs excerpt ==\n'
cat -n rust/src/grok_costs.rs | sed -n '85,120p'
printf '\n== search for project_from_cwd ==\n'
rg -n "project_from_cwd" rust/src
printf '\n== cost_scanner excerpt ==\n'
cat -n rust/src/cost_scanner.rs | sed -n '1,220p'Repository: tsouth89/ceiling
Length of output: 16310
🏁 Script executed:
set -euo pipefail
cat -n rust/src/grok_costs.rs | sed -n '90,110p'
rg -n "project_from_cwd" rust/src
cat -n rust/src/cost_scanner.rs | sed -n '1,220p'Repository: tsouth89/ceiling
Length of output: 11871
Use the shared cwd-to-project helper here. Path::new(path).file_name() only splits on \ on Windows, so a Windows-style cwd stays unsplit on non-Windows hosts. project_from_cwd already handles Windows, UNC, and POSIX paths; reuse it to keep this parser consistent with the rest of the scanner.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@rust/src/grok_costs.rs` around lines 95 - 106, Update the cwd-to-project
conversion in the parser to call the shared project_from_cwd helper instead of
constructing a Path and extracting file_name locally. Preserve the existing
optional cwd handling and trimming, and reuse the helper’s support for Windows,
UNC, and POSIX paths.
1 participant
Summary
weekly_label).~/.grok/sessionsupdates.jsonlturn completions for local Charts usage: tokens over time, cache vs fresh input, reasoning tokens + effort tiers, and project rollups.Test plan
cargo test -p codexbar --lib grok_(parser, discovery, report rollup)cargo test --manifest-path apps/desktop-tauri/src-tauri/Cargo.toml grok_token.\scripts\dev.ps1— Charts → Grok shows local usage card with token mix, effort, projects; bar label reads WeeklyNotes
Release train next: squash to main, then prepare 1.5.6 + signed draft.
Summary by CodeRabbit
New Features
Improvements
Documentation