Skip to content

Mirror the TokensUsageRecord wire contract on RunResults#8

Merged
lchoquel merged 2 commits into
devfrom
feature/tokens-usage-record-mirror
Jul 21, 2026
Merged

Mirror the TokensUsageRecord wire contract on RunResults#8
lchoquel merged 2 commits into
devfrom
feature/tokens-usage-record-mirror

Conversation

@lchoquel

@lchoquel lchoquel commented Jul 21, 2026

Copy link
Copy Markdown
Member

Types the run-usage payload against the wire contract specified in docs/specs/pipelex-mthds-protocol.md → TokensUsage records on run artifacts. Server side shipped in pipelex 0.40.0 / pipelex-api 0.10.0 and has been live on hosted dev since 2026-07-20; this is the client mirror.

What changed

  • TokensUsageRecord (pipelex_sdk/runs.py) — one record per inference call. RunResults.tokens_usages was a list[dict[str, Any]] bag; it now validates into these. Token counts by category, the server-computed cost in USD, model name/id, the pipe that made the call, job-kind fields and timing.
  • Old artifacts still parse. Every field is optional and the model is extra="allow", because durable artifacts written before the contract shipped are relayed verbatim and never migrated: such a record comes back with cost/pipe_code as None and its legacy job_metadata/unit_costs in model_extra.
  • Enum-ish fields stay plain str. model_type, job_category, unit_job_id are open sets on the wire per the spec — deliberately not StrEnum, so runtime enum churn is non-breaking for consumers.
  • Both paths feed the same typed records: the durable path off the results body, the blocking path lifted out of the execute response's extension-open pipe_output.
  • Docs: new docs/run-usage.md (field reference, cost/null semantics, old-artifact rules), cross-linked from docs/architecture.md.

Note the rate table (unit_costs) no longer crosses the wire — a record carries the computed cost instead, None when the model has no rate table at all and 0 when one priced the call at zero.

Breaking

RunResults.pipe_output is now DictPipeOutputAbstract | None, was dict[str, Any] | None. The blocking path already parsed the protocol model and then discarded the types via .model_dump(); it now carries the parsed model through. Read the working memory as attributes — result.pipe_output.working_memory.root["out"].content — rather than nested dict keys. The durable path still leaves it None.

Per the repo convention the version is untouched: the release skill owns the bump and the [Unreleased][vX.Y.Z] conversion. The breaking entry sits under ### Changed, which steers that release to a minor.

Test plan

  • make agent-check — pyright + mypy clean.
  • make agent-test — green. New coverage in tests/unit/test_runs.py (every contract field, pre-contract tolerance, unrated-cost null vs priced-at-zero, the three null branches), plus the durable and blocking client tests updated to assert typed records.

Downstream

The conformance SDK-mirror PR must merge after this one. Its CI installs this package at ref: dev, so merging it first would install a build without TokensUsageRecord and turn that job red.

🤖 Generated with Claude Code


Summary by cubic

Types RunResults.tokens_usages against the wire TokensUsageRecord and exposes usage consistently on both hosted and blocking paths. Also switches RunResults.pipe_output to the typed DictPipeOutputAbstract.

  • New Features

    • Added TokensUsageRecord and RunResults.usage_assembly_error; RunResults.tokens_usages now validates into list[TokensUsageRecord] | None.
    • Populates from both surfaces: hosted tokens_usages.json or blocking pipe_output extras; server-computed cost is included (None = no rate table, 0 = priced zero). No run-level aggregate; sum records.
    • Back-compat: all fields optional and extension-open for old artifacts; tokens_usages is None when usage is off/broke/pre-artifact and [] when no inference ran. Use usage_assembly_error to tell failures. Docs: docs/run-usage.md.
  • Migration

    • Breaking: RunResults.pipe_output is now DictPipeOutputAbstract | None (was dict[str, Any] | None). Read working memory as attributes, e.g. result.pipe_output.working_memory.root["out"].content.
    • Server support shipped in pipelex 0.40.0 / pipelex-api 0.10.0. Adjust any code that depended on a dict-shaped pipe_output.

Written for commit a0418ef. Summary will update on new commits.

Review in cubic

lchoquel and others added 2 commits July 18, 2026 17:28
Both paths populate them: the hosted durable path reads the pair
GET /v1/runs/{id}/results now returns (unpacked from the runner's
tokens_usages.json artifact — pipelex-platform PR #82), and the blocking
fallback unpacks the same pair from the execute response's
extension-open pipe_output, so result.tokens_usages reads the same
regardless of which path ran. None against older platforms or
pre-artifact runs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`RunResults.tokens_usages` was a `list[dict[str, Any]]` bag. It now validates
into `TokensUsageRecord`, a client-side mirror of the wire contract specified in
docs/specs/pipelex-mthds-protocol.md — every field optional and `extra="allow"`,
so pre-contract durable artifacts (relayed verbatim, never migrated) still parse
with `cost`/`pipe_code` None and their legacy `job_metadata`/`unit_costs` kept in
`model_extra`. Enum-ish fields stay plain `str` so runtime enum churn is
non-breaking.

Both paths feed the same typed records: the durable path off the results body,
the blocking path lifted out of the execute response's extension-open
`pipe_output`.

Breaking: `RunResults.pipe_output` is now `DictPipeOutputAbstract | None` rather
than `dict[str, Any] | None`. The blocking path already parsed the protocol
model and then discarded the types via `.model_dump()`; it now carries the parsed
model through, so consumers read the working memory as attributes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Jul 21, 2026

Copy link
Copy Markdown

Greptile Summary

This PR mirrors the run-usage wire contract in the Python SDK. The main changes are:

  • Adds typed, extension-open token usage records.
  • Exposes usage records and assembly errors on run results.
  • Preserves the typed protocol output on blocking runs.
  • Documents current, legacy, null, and cost semantics.

Confidence Score: 5/5

This looks safe to merge.

No blocking issues found in the changed code. Both result paths validate usage records into the same model. Legacy fields remain available through allowed extensions. Tests cover the documented compatibility and null semantics.

T-Rex T-Rex Logs

What T-Rex did

  • Ran the Python SDK run usage agent test to exercise the target suite and verify the contract; the run completed with exit code 0.
  • Compared the parent revision against the before-state and confirmed 42 tests passed with exit 0.
  • Compared the HEAD after changes against the baseline with verbose changed-path output and confirmed 50 tests passed with exit 0.

View all artifacts

T-Rex Ran code and verified through T-Rex

Important Files Changed

Filename Overview
pipelex_sdk/runs.py Adds the usage-record wire model and typed run-result fields while preserving legacy extensions.
pipelex_sdk/client.py Carries typed blocking output through and lifts usage extensions into the shared result fields.
tests/unit/test_runs.py Covers current and legacy records, cost semantics, and null-versus-empty usage states.
tests/unit/test_client_run_fallback.py Covers typed blocking output and usage extraction from protocol extensions.
tests/unit/test_client_lifecycle.py Covers typed usage records returned through the durable lifecycle path.
docs/run-usage.md Documents the usage fields, compatibility rules, and cost and null semantics.

Reviews (1): Last reviewed commit: "feat(runs)!: mirror the TokensUsageRecor..." | Re-trigger Greptile

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 8 files

You’re at about 97% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.

Re-trigger cubic

@lchoquel
lchoquel merged commit 28e9b06 into dev Jul 21, 2026
19 of 20 checks passed
@lchoquel
lchoquel deleted the feature/tokens-usage-record-mirror branch July 21, 2026 21:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant