feat: expose operation usage metadata

[shrey150]

Summary

• expose required per-operation LLM usage metadata on act, observe, and extract results
• default every usage counter to 0, so deterministic actions and cache hits return a complete zero-valued aggregate instead of omitting usage
• aggregate usage across every model inference performed by an operation
• regenerate the canonical JSON Schema and generated Python and Go models
• verify snake_case JSON-RPC transport and camelCase TypeScript results through the real browser runtime

Result shape

result.metadata.usage: {
  inputTokens: number;
  outputTokens: number;
  reasoningTokens: number;
  cachedInputTokens: number;
  inferenceTimeMs: number;
};

The raw JSON-RPC response uses input_tokens, output_tokens, reasoning_tokens, cached_input_tokens, and inference_time_ms. All five fields are always present. Operations that do not run inference return 0 for all five fields.

Usage is scoped to one operation and aggregates every model call made by that operation. Session-wide aggregation through stagehand.metrics() is intentionally outside this PR.

E2E Test Matrix

Command / flow	Observed output	Confidence / sufficiency
Local built SDK and server with headless Chromium and a real Groq model: extract → observe → act against https://example.com	All three operations succeeded and returned metadata.usage; extract returned 811 input / 237 output tokens, observe returned 680 / 376, and act returned 1149 / 313 and navigated to the expected public destination	Proves real provider inference reaches each operation result through the locally built runtime. It covers one provider/model, not every supported provider.
node_modules/.bin/vitest run --root . packages/sdk-ts/tests/browser-runtime/stagehand-launch-connect-smoke.test.ts	1 browser-runtime file and 13 tests passed in real headless Chromium; deterministic act returned five zero-valued camelCase SDK fields and five zero-valued snake_case wire fields	Proves the required zero aggregate crosses the actual extension/service-worker JSON-RPC transport and is decoded by the public SDK. The test model is deterministic so exact aggregation can be asserted.
<Corepack PATH> node_modules/.bin/turbo run test:unit	9 tasks passed: protocol 321, TypeScript SDK 117, server 197 with 11 todo, root 72, docs 18, and evals 403	Covers schema defaults, required result fields, multi-inference aggregation, self-healing actions, cache hits, deterministic actions, generated models, and package installation.
node_modules/.bin/turbo run build && node_modules/.bin/turbo run fmt:check lint typecheck	Build completed; all 8 check tasks passed with no errors	Covers package builds, formatting, lint, type checking, and documentation validation.
uv --directory packages/sdk-python run --locked pytest plus Ruff, Ty, and generator checks	208 tests passed; formatting, lint, type, and generated-model checks passed	Proves the generated Python model exposes the complete required usage result and remains synchronized with the protocol.
go list ./... package tests plus generator tests/check	All non-example Go packages and the generator passed	Proves the generated Go model and embedded extension match the merged protocol/server source.

Follow-up

A stacked PR will implement the existing stagehand.metrics() RPC using V3-style session totals built from these per-operation usage values.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 99d58ca

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[cubic-dev-ai]

cubic analysis

No issues found across 16 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

Linked issue analysis

Linked issue: STG-2670: figure out what we are doing with .metrics & token tracking

Status	Acceptance criteria	Notes
✅	Add optional per-operation usage metadata to act/observe/extract results with fields inputTokens, outputTokens, reasoningTokens, cachedInputTokens, inferenceTimeMs	Schema, docs, and runtime all add the usage object and the specified fields; tests assert presence/shape in SDK and server results.
✅	Populate usage from real model inference and aggregate multi-call operations (e.g., two-step actions)	Server code converts inference responses into usage, aggregates multi-call usage, and tests verify aggregation and numeric totals.
✅	Omit usage metadata when an operation did not run inference (deterministic actions / cache hits)	Runtime preserves omission for operations without inference and tests assert usage is undefined for cache hits or purely deterministic actions.
✅	Ensure JSON-RPC wire uses snake_case and SDK surfaces camelCase fields (round-trip casing + verify through browser runtime)	Wire-casing tests and browser-runtime smoke test capture raw JSON-RPC messages and assert snake_case on the wire and camelCase in SDK results; test instruments transport to check raw messages.
✅	Regenerate canonical JSON Schema and update generated Python and Go models to include the usage shape	The canonical JSON schema and generated model files were updated to include StagehandResultUsage and the checked-in generated models reflect the new fields.

Architecture diagram

sequenceDiagram
    participant Client as Client SDK
    participant Server as RPC Server
    participant ActSvc as Act Service
    participant LLM as LLM Service
    participant Cache as Cache Store

    Note over Client,Cache: 🆕 Per-operation LLM usage metadata

    Client->>Server: act({ instruction, page })
    Server->>ActSvc: act(params)

    alt instruction is Action (deterministic)
        ActSvc->>ActSvc: performAction(deterministic)
        Note over ActSvc: No LLM inference
        ActSvc-->>Server: { data: { success, ... }, metadata: {} }
    else instruction is a string
        ActSvc->>Cache: getCachedActions(instruction)
        alt Cache HIT
            Cache-->>ActSvc: cached actions
            Note over ActSvc: No LLM inference
            ActSvc-->>Server: { data: { ... }, metadata: {} }
        else Cache MISS
            ActSvc->>LLM: generate(instruction, model)
            LLM-->>ActSvc: { action, twoStep, usage: { prompt_tokens, ... } }
            ActSvc->>ActSvc: aggregateUsage(initial)
            alt twoStep is true
                ActSvc->>LLM: generate(second_action)
                LLM-->>ActSvc: { action, usage: { ... } }
                ActSvc->>ActSvc: aggregateUsage(additional)
            end
            ActSvc->>ActSvc: performAction()
            ActSvc->>Cache: setCachedActions(actions)
            ActSvc-->>Server: { data: { ... }, metadata: { usage: { inputTokens, ... } } }
        end
    end

    Note over Server,Client: JSON-RPC transport converts to snake_case (input_tokens, etc.)
    Server-->>Client: act result (with optional usage)
    Client->>Client: decode to camelCase

Loading

_{Re-trigger cubic}