Skip to content

Add per-activity and per-workflow token use tracking and cost estimation #232

Description

@m2ux

Problem Statement

Workflow sessions provide no visibility into the token consumption or monetary cost of executing a workflow. Token use cannot be attributed to individual activities, so there is no way to identify expensive activities, compare workflow efficiency, or account for the cost of a completed work package.

Current state:

  • No record of token use is kept for a session, an activity, or a workflow.
  • Cost of a workflow run is unknown and cannot be reviewed after completion.
  • Expensive activities cannot be identified or compared across runs.

Desired state:

  • Token use is tracked per activity and aggregated per workflow.
  • A cost estimate is derived from the tracked usage.
  • The results are logged to the appropriate planning artifact upon workflow completion, so each work package carries its own usage and cost record.

Goal

Enable users to see how many tokens each activity and each workflow run consumes, and what that execution costs.

Scope

In Scope

  • Per-activity token use tracking during a workflow session.
  • Per-workflow aggregation of tracked usage.
  • Cost estimation derived from tracked usage.
  • Logging the results to the appropriate planning artifact on completion.

Out of Scope

  • Cross-session or fleet-wide usage analytics (a future concern once per-run tracking exists).
  • Budget enforcement or hard limits on token use (tracking and reporting only).

User Stories

US-1: Per-activity visibility

As a workflow operator, I want token use recorded per activity so that I can identify which activities are expensive.

Acceptance Criteria:

  • Each completed activity in a session has an associated token-use record.

US-2: Per-workflow totals and cost

As a workflow operator, I want per-workflow token totals and a cost estimate so that I know what a work package cost to execute.

Acceptance Criteria:

  • A completed workflow run reports aggregate token use across its activities.
  • A cost estimate is derived from the aggregate usage.

US-3: Durable record

As a work-package stakeholder, I want the usage and cost results logged to the planning artifact upon completion so that the record persists with the work package.

Acceptance Criteria:

  • On completion, the per-activity and per-workflow results appear in the appropriate planning artifact.

Constraints

  • Measurement should use the model's/harness's native usage-reporting capabilities where possible, rather than heuristic estimation.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions