Total Costs/Token Usage across all Tasks

[jmasarweh]

What specific problem does this solve?

Currently, tasks cost usage is only visible per task with no visibility of the total usage per workspace or across all workspaces. I have forked and created a suggestive solution implementing an "AI Inference Summary" section in the welcome/chat section. It breaks down the usage per dollar spent and per input/output token (+cached). It also categorizes the breakdown per ROO Mode. This was much needed to keep track spend across all AI providers, instead of doing the calculation per task and manually, or checking each provider's billing pages.

Teams and users will have a clearer visibility on Cost and Token usage across work spaces.

The suggested implementation uses an async call to wait for all Roo screen elements and providers to load before adding the AI Inference Summary. There shouldn't be any impact on exisiting features.

Working demo at: Roo AI-Inference-Summary -- just Run Start Debug (F5)

Additional context (optional)

All Workspaces Summary:

Current Workspace Summary:

Full Screen View:

Roo Code Task Links (Optional)

No response

Request checklist

• I've searched existing Issues and Discussions for duplicates
• This describes a specific problem with clear impact and context

Interested in implementing this?

• Yes, I'd like to help implement this feature

Implementation requirements

• I understand this needs approval before implementation begins

How should this be solved? (REQUIRED if contributing, optional otherwise)

Core Features Implemented

1. AI Inference Summary Component
   Added a comprehensive cost summary display showing total cost, input/output tokens, and cache breakdown
   Positioned prominently on the landing screen with responsive design
   Includes mode breakdown with individual costs and token counts for each AI mode used

2. Persistent Storage Solution
   Created a robust backend service that stores usage data in ~/.roo/usage-tracking.json
   Solves the critical issue where deleting tasks from history also deleted their associated costs
   Uses atomic write operations with file locking to prevent data corruption
   Automatically migrates existing task history data to the persistent storage

3. Workspace Filtering
   Added toggle buttons in the top-right corner of the AI Inference Summary card
   Two views: "Current" (current workspace only) and "All" (all workspaces)
   Dynamic workspace name display showing the folder name
   Proper state management for filter switching

4. Backend Integration
   Integrated the service into ClineProvider.ts to track all history changes
   Provides both workspace views to the frontend asynchronously
   Handles data loading efficiently without blocking main UI updates

5. Comprehensive Testing
   Backend service has full test coverage (all tests passing)
   Tests cover data persistence, workspace filtering, error handling, and edge cases

Key Technical Details
File Location: usage-tracking.json stored in global ~/.roo/ directory (outside workspace)
Data Persistence: Survives task deletions, VSCode restarts, and workspace changes
Performance: Asynchronous data loading doesn't block main UI rendering
Reliability: Atomic writes with error handling prevent data loss
Workspace Scoping: Correctly filters data by workspace path for accurate per-project tracking
Added Test Coverage.

Files Changed for AI Inference Summary Feature

New Files Created:

packages/types/src/usage.ts

• Defines PersistentUsageItem schema with Zod validation for storing cost/token data
• Includes taskWorkspace field for workspace filtering and removes taskTitle for storage efficiency

src/services/usage-tracking/index.ts

• Backend service for persistent usage data management in ~/.roo/usage-tracking.json
• Provides updateUsageTracking() and getUsageSummary() functions with workspace filtering support

src/services/usage-tracking/__tests__/index.spec.ts

• Comprehensive unit tests for the usage tracking service
• Tests data persistence, workspace filtering, and error handling scenarios

webview-ui/src/components/welcome/CostSummary.tsx

• React component displaying total costs, tokens, and breakdown by mode
• Includes toggle buttons for switching between "All Workspaces" and "Current Workspace" views

webview-ui/src/components/welcome/__tests__/CostSummary.spec.tsx

• Unit tests for the CostSummary React component
• Tests component rendering, state management, and workspace filtering functionality

Modified Files:

packages/types/src/index.ts

• Added exports for new usage-related types (PersistentUsageItem, UsageSummary)

src/shared/ExtensionMessage.ts

• Added persistentUsageData message type for webview communication
• Extended message protocol to support workspace-filtered usage data

src/core/webview/ClineProvider.ts

• Integrated usage tracking service calls on task history updates
• Added loadPersistentUsageDataAsync() method for asynchronous data loading

webview-ui/src/components/welcome/WelcomeView.tsx

• Imported and rendered the new CostSummary component
• Added component to the welcome screen layout

webview-ui/src/context/ExtensionStateContext.tsx

• Added persistentUsageData state to context for usage summary data
• Added message handler for persistentUsageData messages from backend

Summary

• 5 new files for persistent usage tracking and UI component
• 5 modified files for integration and state management
• All changes maintain backward compatibility with existing functionality

How will we know it works? (Acceptance Criteria - REQUIRED if contributing, optional otherwise)

Acceptance Criteria

Basic Display Functionality
Given I open the Roo Code extension for the first time

When I view the welcome screen

Then I see an "AI Inference Summary" section displaying total cost, total tokens, and breakdown by mode

And the summary shows "All Workspaces" data by default

And all cost values are formatted with proper currency symbols

But no sensitive information like API keys are displayed

Workspace Filtering
Given I have tasks completed in multiple workspaces

When I click the "Current Workspace" toggle button

Then the summary updates to show only data for the current workspace

And the toggle button shows the active selection visually

And I can switch back to "All Workspaces" view

But the UI doesn't flicker or show loading states during the switch

Data Persistence
Given I have completed several tasks with various costs and token usage

When I delete a task from the task history

Then the AI Inference Summary still includes the cost and token data from that deleted task

And the summary totals remain accurate

And the data persists across VSCode restarts

But the deleted task no longer appears in the task history list

Real-time Updates
Given I have the welcome screen open

When I complete a new task with API usage

Then the AI Inference Summary automatically updates to include the new usage data

And the totals reflect the new task's cost and tokens

And the mode breakdown shows the updated statistics

But the update happens without requiring a page refresh

Multi-Mode Breakdown
Given I have completed tasks using different modes (Code, Debug, Ask, etc.)

When I view the AI Inference Summary

Then I see a breakdown showing cost and token usage for each mode I've used

And each mode shows both input and output token counts

And the individual mode totals sum up to the overall totals

But modes I haven't used don't appear in the breakdown

Error Handling
Given the persistent usage tracking file is corrupted or missing

When I view the welcome screen

Then the AI Inference Summary shows zero values or a loading state

And the component doesn't crash or show error messages to the user

And new task completions still get tracked properly

But the extension continues to function normally

Large Dataset Performance
Given I have completed over 100 tasks with extensive API usage

When I switch between workspace views

Then the filtering and display updates happen within 1 second

And the interface remains responsive during data processing

And all calculations remain accurate

But the browser doesn't freeze or become unresponsive

These acceptance criteria cover the core functionality, edge cases, and performance expectations for the AI Inference Summary feature.

Technical considerations (REQUIRED if contributing, optional otherwise)

Technical Considerations

Implementation Approach

• Persistent Storage Architecture: Uses a JSON file (usage-tracking.json) in the global ~/.roo/ directory to maintain data independence from task lifecycle
• Service Layer: New src/services/usage-tracking/ service handles all persistence operations with atomic file writes using safeWriteJson utility
• Dual Data Sources: Maintains both volatile task history (existing) and persistent usage data (new) for different use cases
• Asynchronous Loading: Usage data loads independently after main state to prevent blocking UI initialization
• Schema Evolution: New PersistentUsageItem schema with workspace tracking and deduplicated storage

Architecture Changes

• Type System Updates: New interfaces in packages/types/src/usage.ts with Zod validation schemas
• Message Protocol: Extended ExtensionMessage.ts with persistentUsageData message type for webview communication
• Provider Integration: ClineProvider.ts now orchestrates both task history and usage tracking updates
• Component Architecture: New CostSummary.tsx React component with local state management for filtering
• Data Flow: Backend calculates summaries, frontend handles presentation and user interaction

Performance Implications

• File I/O Operations: JSON file reads/writes on task completion (~1-5ms for typical datasets)
• Memory Footprint: Additional ~100KB RAM for 1000 tasks worth of usage data
• Calculation Complexity: O(n) filtering operations where n = number of completed tasks
• UI Responsiveness: Asynchronous data loading prevents blocking main thread
• Debouncing: UI state changes are immediate, no debouncing needed for simple toggle operations

Compatibility Concerns

• Cross-Platform: File system operations tested on Windows/macOS/Linux
• VSCode API: Uses standard extension context and file system APIs (no experimental features)
• React Versions: Compatible with existing React 18+ and TypeScript 4.5+ requirements
• Existing Data: Backwards compatible with existing taskHistory structure
• Migration: Automatic one-time migration of existing tasks to persistent storage

Systems Affected

• Task Lifecycle Management: Hooks into task creation, completion, and deletion events
• Global State System: Extends ContextProxy usage without modifying existing patterns
• Webview Messaging: Adds new message type while maintaining existing communication patterns
• File System Layer: New dependency on ~/.roo/ directory structure and file permissions
• UI State Management: New React component integrates with existing ExtensionStateContext

Potential Blockers

• File Permissions: Users with restricted file system access to home directory could face issues
• Concurrent Access: Multiple VSCode instances could create race conditions (mitigated by atomic writes)
• Storage Limitations: Very heavy users (10,000+ tasks) might experience slower file operations
• Migration Complexity: Edge cases in migrating existing task history with missing cost/token data
• Testing Coverage: Complex interaction between persistent and volatile data sources requires thorough integration testing
• Memory Constraints: Large datasets could impact extension memory usage in resource-constrained environments

Risk Mitigation

• Graceful fallback to zero values if persistent storage fails
• Error boundaries prevent UI crashes from data loading issues
• Atomic file operations with backup/restore capability
• Comprehensive unit and integration test coverage
• Performance monitoring for large dataset scenarios

Trade-offs and risks (REQUIRED if contributing, optional otherwise)

Potential Blockers

File Permissions: Users with restricted file system access to home directory could face issues
Concurrent Access: Multiple VSCode instances could create race conditions (mitigated by atomic writes)
Storage Limitations: Very heavy users (10,000+ tasks) might experience slower file operations
Migration Complexity: Edge cases in migrating existing task history with missing cost/token data
Testing Coverage: Complex interaction between persistent and volatile data sources requires thorough integration testing
Memory Constraints: Large datasets could impact extension memory usage in resource-constrained environments

Risk Mitigation

Graceful fallback to zero values if persistent storage fails
Error boundaries prevent UI crashes from data loading issues
Atomic file operations with backup/restore capability
Comprehensive unit and integration test coverage
Performance monitoring for large dataset scenarios

[roomote]

Hi! I'm here to help out the maintainers and am going to work on implementing this AI Inference Summary feature. I can see you've already provided a comprehensive implementation plan with detailed specifications. I'll follow your outlined approach to add persistent usage tracking and the cost summary display. I'll keep you updated on the progress!

[daniel-lxs]

Hey @jmasarweh I like this proposal, I'll go ahead and assign the issue to you