feat(core): expose AI SDK usage contract in agent hooks#1372
Conversation
🦋 Changeset detectedLatest commit: ef12f66 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
📝 WalkthroughWalkthroughThis PR updates agent ChangesAI SDK usage contract update
Estimated code review effort: 3 (Moderate) | ~30 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Deploying voltagent with
|
| Latest commit: |
ef12f66
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://36da4ffe.voltagent.pages.dev |
| Branch Preview URL: | https://feat-ai-sdk-usage-contract.voltagent.pages.dev |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
packages/core/src/agent/agent.ts (1)
1698-1713: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winBail-out
onEndpayload omitssteps, unlike every other path.The normal-completion
onEndpayloads for generateText/streamText/generateObject/streamObject all includesteps, but this bail-out path does not — even thoughoc.systemContext.get("conversationSteps")holds the steps that occurred before the bail (used just below atrecordStepResults(undefined, oc)). Consumers relying ononEnd.output.stepsfor context-window/usage bookkeeping in a bailed multi-step run will silently getundefined.🐛 Suggested fix
await this.getMergedHooks(options).onEnd?.({ conversationId: oc.conversationId || "", agent: this, output: { text: finalText, usage: providerUsage, totalUsage: providerUsage, + steps: oc.systemContext.get("conversationSteps") as + | ReadonlyArray<StepResult<ToolSet>> + | undefined, providerResponse: undefined as any, finishReason: "bail" as any, warnings: undefined, context: oc.context, },🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/core/src/agent/agent.ts` around lines 1698 - 1713, The bail-out onEnd payload in Agent’s end-of-run flow is missing steps, unlike the normal completion paths. Update the onEnd call in agent.ts to include the conversation steps from oc.systemContext.get("conversationSteps") in output.steps, matching the payload shape used by generateText/streamText/generateObject/streamObject. Use the surrounding oc, recordStepResults(undefined, oc), and this.getMergedHooks(options).onEnd?.() code to locate the bail-out branch and ensure the steps are passed through consistently..changeset/ai-sdk-usage-contract.md (1)
1-6: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winBump
@voltagent/coretomajorThis is a breaking API change:
usagechanges semantics and shape, so the changeset should not beminor.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.changeset/ai-sdk-usage-contract.md around lines 1 - 6, Update the changeset for `@voltagent/core` from minor to major because the agent onEnd output contract is changing in a breaking way: usage now means final-step usage, totalUsage is the aggregate, and steps may be included. Adjust the changeset entry in ai-sdk-usage-contract.md so the version bump matches this API-breaking semantics change.
🧹 Nitpick comments (2)
packages/core/src/agent/agent.ts (2)
3666-3687: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value
streamObject's returned result wrapper doesn't exposetotalUsage/steps, unlikestreamText's.
streamText'sresultWithContext(line 2819-2824) addstotalUsage/stepsgetters, butstreamObject's equivalent wrapper here doesn't, so callers only get these fields via theonEndhook, not on the returned object. GivenstreamObjectis already@deprecated, this is likely low priority to backfill.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/core/src/agent/agent.ts` around lines 3666 - 3687, The streamObject result wrapper in agent.ts is missing the same totalUsage and steps exposure that streamText’s resultWithContext provides. Update the streamObject wrapper so its returned object delegates these fields via getters, alongside the existing partialObjectStream/textStream/warnings/usage/finishReason properties, using the same pattern as the streamText wrapper and keeping the implementation anchored around resultWithContext and StreamObjectResultWithContext.
3125-3127: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winRepeated unsafe
as {...}casts fortotalUsage/steps— consider a shared type.
generateObject/streamObjectresults are repeatedly cast inline ((result as { totalUsage?: LanguageModelUsage }).totalUsage,(finalResult as { steps?: ReadonlyArray<StepResult<ToolSet>> }).steps) because the underlying AI SDK result types don't natively expose these fields. This bypasses structural type-checking in 4+ places and duplicates the same ad-hoc shape.As per coding guidelines, "Maintain type safety in TypeScript-first codebase."
♻️ Suggested consolidation
+type ResultWithUsageMeta<T> = T & { + totalUsage?: LanguageModelUsage; + steps?: ReadonlyArray<StepResult<ToolSet>>; +};Then reuse
ResultWithUsageMeta<typeof result>/ResultWithUsageMeta<typeof finalResult>at each of the 4 call sites instead of repeating the inline cast shape.Also applies to: 3560-3562
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/core/src/agent/agent.ts` around lines 3125 - 3127, The repeated inline unsafe casts for `totalUsage` and `steps` in `agent.ts` should be replaced with a shared type to preserve TypeScript safety and avoid duplicating ad-hoc shapes. Introduce and reuse a helper like `ResultWithUsageMeta<typeof result>` / `ResultWithUsageMeta<typeof finalResult>` at the `generateObject` and `streamObject` result handling sites, including the `result` and `finalResult` assignments, so the same metadata fields are accessed through a consistent typed shape rather than repeated `(result as {...})` casts.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In @.changeset/ai-sdk-usage-contract.md:
- Around line 1-6: Update the changeset for `@voltagent/core` from minor to major
because the agent onEnd output contract is changing in a breaking way: usage now
means final-step usage, totalUsage is the aggregate, and steps may be included.
Adjust the changeset entry in ai-sdk-usage-contract.md so the version bump
matches this API-breaking semantics change.
In `@packages/core/src/agent/agent.ts`:
- Around line 1698-1713: The bail-out onEnd payload in Agent’s end-of-run flow
is missing steps, unlike the normal completion paths. Update the onEnd call in
agent.ts to include the conversation steps from
oc.systemContext.get("conversationSteps") in output.steps, matching the payload
shape used by generateText/streamText/generateObject/streamObject. Use the
surrounding oc, recordStepResults(undefined, oc), and
this.getMergedHooks(options).onEnd?.() code to locate the bail-out branch and
ensure the steps are passed through consistently.
---
Nitpick comments:
In `@packages/core/src/agent/agent.ts`:
- Around line 3666-3687: The streamObject result wrapper in agent.ts is missing
the same totalUsage and steps exposure that streamText’s resultWithContext
provides. Update the streamObject wrapper so its returned object delegates these
fields via getters, alongside the existing
partialObjectStream/textStream/warnings/usage/finishReason properties, using the
same pattern as the streamText wrapper and keeping the implementation anchored
around resultWithContext and StreamObjectResultWithContext.
- Around line 3125-3127: The repeated inline unsafe casts for `totalUsage` and
`steps` in `agent.ts` should be replaced with a shared type to preserve
TypeScript safety and avoid duplicating ad-hoc shapes. Introduce and reuse a
helper like `ResultWithUsageMeta<typeof result>` / `ResultWithUsageMeta<typeof
finalResult>` at the `generateObject` and `streamObject` result handling sites,
including the `result` and `finalResult` assignments, so the same metadata
fields are accessed through a consistent typed shape rather than repeated
`(result as {...})` casts.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 626b0c92-b0dc-4648-ba61-b395b4c02523
📒 Files selected for processing (7)
.changeset/ai-sdk-usage-contract.mdpackages/core/src/agent/agent.spec-d.tspackages/core/src/agent/agent.spec.tspackages/core/src/agent/agent.tspackages/core/src/agent/hooks/index.spec.tspackages/core/src/agent/subagent/test-utils.tspackages/core/src/agent/types.ts
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@website/docs/agents/hooks.md`:
- Around line 625-632: The per-step usage example in the hooks docs assumes
every entry from output.steps has usage, which can throw when a step omits it.
Update the example around finalStepUsage, aggregateUsage, and perStepUsage so it
safely handles optional step usage by filtering or guarding before reading
usage.totalTokens, and keep the logging example aligned with the
output.steps?.map usage contract.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 6f48b1d2-2acf-4fe1-9046-ff07367009e1
📒 Files selected for processing (3)
website/docs/agents/context.mdwebsite/docs/agents/hooks.mdwebsite/docs/agents/overview.md
✅ Files skipped from review due to trivial changes (1)
- website/docs/agents/context.md
| const finalStepUsage = output.usage; | ||
| const aggregateUsage = output.totalUsage; | ||
| const perStepUsage = output.steps?.map((step) => step.usage); | ||
|
|
||
| console.log({ | ||
| finalStepTokens: finalStepUsage?.totalTokens, | ||
| aggregateTokens: aggregateUsage?.totalTokens, | ||
| stepTokens: perStepUsage?.map((usage) => usage.totalTokens), |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Guard the per-step usage example against missing data.
output.steps is optional, and the new contract only guarantees steps[].usage when it is available. As written, usage.totalTokens will throw if any step omits usage.
Proposed fix
- const perStepUsage = output.steps?.map((step) => step.usage);
+ const perStepUsage = output.steps?.flatMap((step) => (step.usage ? [step.usage] : []));📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const finalStepUsage = output.usage; | |
| const aggregateUsage = output.totalUsage; | |
| const perStepUsage = output.steps?.map((step) => step.usage); | |
| console.log({ | |
| finalStepTokens: finalStepUsage?.totalTokens, | |
| aggregateTokens: aggregateUsage?.totalTokens, | |
| stepTokens: perStepUsage?.map((usage) => usage.totalTokens), | |
| const finalStepUsage = output.usage; | |
| const aggregateUsage = output.totalUsage; | |
| const perStepUsage = output.steps?.flatMap((step) => (step.usage ? [step.usage] : [])); | |
| console.log({ | |
| finalStepTokens: finalStepUsage?.totalTokens, | |
| aggregateTokens: aggregateUsage?.totalTokens, | |
| stepTokens: perStepUsage?.map((usage) => usage.totalTokens), |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@website/docs/agents/hooks.md` around lines 625 - 632, The per-step usage
example in the hooks docs assumes every entry from output.steps has usage, which
can throw when a step omits it. Update the example around finalStepUsage,
aggregateUsage, and perStepUsage so it safely handles optional step usage by
filtering or guarding before reading usage.totalTokens, and keep the logging
example aligned with the output.steps?.map usage contract.
1 participant
PR Checklist
Please check if your PR fulfills the following requirements:
Bugs / Features
What is the current behavior?
Agent
onEndoutput exposesoutput.usageas the aggregate run usage after #1366. That keeps provider semantics consistent, but it hides the AI SDK final-step usage contract from hook consumers. Consumers that need context-window accounting for the most recent step have to infer or reconstruct it themselves.What is the new behavior?
Agent
onEndoutputs now expose AI SDK usage semantics directly:output.usageis the final-step AI SDKresult.usage.output.totalUsageis the aggregate AI SDKresult.totalUsage.output.stepscarries AI SDK step results when available, includingsteps[].usage.Internal observability, persistence, middleware, and guardrail metadata still use the aggregate usage path so trace/root usage remains cumulative and provider-consistent.
fixes #1368
Notes for reviewers
This intentionally changes the public
onEnd.output.usagemeaning to match AI SDK v6 semantics. A regression test covers a 3-step run where final-step usage is222, aggregate usage is508, andsteps[].usageexposes each individual step.Validation run:
pnpm --filter @voltagent/core typecheckpnpm --filter @voltagent/core test:single src/agent/hooks/index.spec.ts src/agent/agent.spec.ts src/agent/subagent/index.spec.ts src/agent/subagent/bail.spec.tspnpm --filter @voltagent/core lintpnpm exec prettier --check website/docs/agents/hooks.md website/docs/agents/overview.md website/docs/agents/context.mdpnpm --dir website buildThe core lint command exits successfully but reports pre-existing complexity warnings in
agent.ts,workflow/steps/and-agent.ts, andworkspace/filesystem/backends/filesystem.ts. Docs were added underwebsite/docs/agents/hooks.md, with related examples updated inwebsite/docs/agents/overview.mdandwebsite/docs/agents/context.md.pnpm --dir website lintwas also checked and currently fails on unrelated existing website issues (website/src/data/tweets.jsonformatting,PricingCalculatorModal.tsx,Testimonials.tsx, andsupervisor-agent/flow-version.tsxexhaustive deps).Summary by cubic
Expose AI SDK v6 usage semantics in agent hooks and streaming results so
onEndreports final-step usage, with aggregate and per-step details when available. Docs now show how to read final-step vs total usage in hooks and streams.New Features
@voltagent/corehooks,output.usageis the final-step AI SDKresult.usage.output.totalUsageexposes the aggregate AI SDKresult.totalUsage.output.stepsincludes AI SDK step results (withsteps[].usage) when available.totalUsageandstepsalongsideusage.finalStepUsagevstotalUsagein hooks and streaming examples.Migration
output.usage, switch tooutput.totalUsage.usagefields now use the AI SDKLanguageModelUsageshape.Written for commit ef12f66. Summary will update on new commits.
Summary by CodeRabbit
onEndcallbacks and stream results now include richer token-usage reporting: final-stepusage, aggregatetotalUsage, andstepswith per-step usage when available.output.totalUsageand detailed per-step usage.