Skip to content

fix(ai): preserve tool result provider options#15186

Closed
Genmin wants to merge 1 commit into
vercel:mainfrom
Genmin:fix/tool-model-output-provider-options
Closed

fix(ai): preserve tool result provider options#15186
Genmin wants to merge 1 commit into
vercel:mainfrom
Genmin:fix/tool-model-output-provider-options

Conversation

@Genmin
Copy link
Copy Markdown
Contributor

@Genmin Genmin commented May 12, 2026

Summary

Root cause

toModelOutput() can return provider options on the ToolResultOutput, but the generated tool-result parts only carried existing part-level provider metadata. Provider adapters such as Anthropic read cache-control settings from the tool-result part, so metadata returned by toModelOutput() was dropped before conversion.

Testing

  • corepack pnpm --filter ai exec vitest --config vitest.node.config.js --run src/generate-text/generate-text.test.ts src/generate-text/to-response-messages.test.ts
  • corepack pnpm --filter ai exec vitest --config vitest.node.config.js --run src/ui/convert-to-model-messages.test.ts
  • corepack pnpm --filter ai type-check
  • corepack pnpm --filter ai build
  • corepack pnpm exec ultracite check .changeset/tool-model-output-provider-options.md packages/ai/src/prompt/get-tool-result-provider-options.ts packages/ai/src/generate-text/generate-text.ts packages/ai/src/generate-text/stream-text.ts packages/ai/src/generate-text/to-response-messages.ts packages/ai/src/ui/convert-to-model-messages.ts packages/ai/src/generate-text/generate-text.test.ts packages/ai/src/generate-text/to-response-messages.test.ts
  • git diff --check

aayush-kapoor
aayush-kapoor reviewed May 14, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@aayush-kapoor aayush-kapoor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not the right fix - it can be contained in the anthropic provider

@aayush-kapoor aayush-kapoor mentioned this pull request May 14, 2026
Merged
5 tasks
aayush-kapoor added a commit that referenced this pull request May 15, 2026
@aayush-kapoor
fix(anthropic): propagate toModelOutput providerOption to anthropic t…
a464505 
…ool results (#15284)

## Background

#15185

when a user used `tool.toModelOutput()` to return some providerOptions,
the problem was that Anthropic never read them from that location.

metadata existed at: `part.output.providerOptions` but the Anthropic
converter only checked: `part.providerOptions`,
`message.providerOptions`

so `cache_control` was silently omitted from the outgoing anthropic
tool_result block

## Summary

converter now also checks provider options on the tool-result output
itself, including inner content output parts

## Manual Verification

verified by running the following repro:

<details>
<summary>repro:</summary>

```ts
import {
  anthropic,
  type AnthropicLanguageModelOptions,
} from '@ai-sdk/anthropic';
import { generateText, isStepCount, tool, type ModelMessage } from 'ai';
import fs from 'node:fs';
import { z } from 'zod';
import { run } from '../../lib/run';

const cachedText = fs.readFileSync('data/error-message.txt', 'utf8');

const anthropicOptions = {
  cacheControl: { type: 'ephemeral' },
} satisfies AnthropicLanguageModelOptions;

const providerOptions = {
  anthropic: anthropicOptions,
};

const controlMessages = [
  {
    role: 'assistant',
    content: [
      {
        type: 'text',
        text: cachedText,
        providerOptions,
      },
    ],
  },
  {
    role: 'user',
    content: 'Reply with "ok".',
  },
] satisfies ModelMessage[];

const cachedTextTool = tool({
  description: 'Return a stable cached text payload.',
  inputSchema: z.object({}),
  execute: async () => cachedText,
  toModelOutput: ({ output }) => ({
    type: 'text',
    value: output,
    providerOptions,
  }),
});

function hasCacheControl(body: unknown) {
  return JSON.stringify(body).includes('"cache_control"');
}

run(async () => {
  const controlCold = await generateText({
    model: anthropic('claude-opus-4-1'),
    messages: controlMessages,
    include: { requestBody: true },
  });

  console.log('=== control cold ===');
  console.log(controlCold.finalStep.providerMetadata?.anthropic?.usage);
  console.log('request has cache_control:', hasCacheControl(controlCold.request.body));

  const controlWarm = await generateText({
    model: anthropic('claude-opus-4-1'),
    messages: controlMessages,
    include: { requestBody: true },
  });

  console.log('=== control warm ===');
  console.log(controlWarm.finalStep.providerMetadata?.anthropic?.usage);
  console.log('request has cache_control:', hasCacheControl(controlWarm.request.body));

  const toolCold = await generateText({
    model: anthropic('claude-opus-4-1'),
    prompt: 'Call the cachedText tool, then reply with "ok".',
    tools: { cachedText: cachedTextTool },
    toolChoice: { type: 'tool', toolName: 'cachedText' },
    stopWhen: isStepCount(2),
    include: { requestBody: true },
  });

  console.log('=== tool result cold ===');
  console.log(toolCold.finalStep.providerMetadata?.anthropic?.usage);
  console.log('request has cache_control:', hasCacheControl(toolCold.finalStep.request.body));

  const toolWarm = await generateText({
    model: anthropic('claude-opus-4-1'),
    messages: [
      { role: 'user', content: 'Call the cachedText tool, then reply with "ok".' },
      ...toolCold.responseMessages,
      { role: 'user', content: 'Reply with "ok" again.' },
    ],
    tools: { cachedText: cachedTextTool },
    include: { requestBody: true },
  });

  console.log('=== tool result warm ===');
  console.log(toolWarm.finalStep.providerMetadata?.anthropic?.usage);
  console.log('request has cache_control:', hasCacheControl(toolWarm.request.body));
});

```
</details>

Observed behavior before the fix is that the last 2 tool-results will
show `request has cache_control: false`

## Checklist

- [x] All commits are signed (PRs with unsigned commits cannot be
merged)
- [x] Tests have been added / updated (for bug fixes / features)
- [ ] Documentation has been added / updated (for bug fixes / features)
- [x] A _patch_ changeset for relevant packages has been added (for bug
fixes / features - run `pnpm changeset` in the project root)
- [x] I have reviewed this pull request (self-review)

## Related Issues

fixes #15186
github-actions Bot added a commit that referenced this pull request May 15, 2026
@github-actions
fix(anthropic): propagate toModelOutput providerOption to anthropic t…
ef16eab 
…ool results (#15284)

## Background

#15185

when a user used `tool.toModelOutput()` to return some providerOptions,
the problem was that Anthropic never read them from that location.

metadata existed at: `part.output.providerOptions` but the Anthropic
converter only checked: `part.providerOptions`,
`message.providerOptions`

so `cache_control` was silently omitted from the outgoing anthropic
tool_result block

## Summary

converter now also checks provider options on the tool-result output
itself, including inner content output parts

## Manual Verification

verified by running the following repro:

<details>
<summary>repro:</summary>

```ts
import {
  anthropic,
  type AnthropicLanguageModelOptions,
} from '@ai-sdk/anthropic';
import { generateText, isStepCount, tool, type ModelMessage } from 'ai';
import fs from 'node:fs';
import { z } from 'zod';
import { run } from '../../lib/run';

const cachedText = fs.readFileSync('data/error-message.txt', 'utf8');

const anthropicOptions = {
  cacheControl: { type: 'ephemeral' },
} satisfies AnthropicLanguageModelOptions;

const providerOptions = {
  anthropic: anthropicOptions,
};

const controlMessages = [
  {
    role: 'assistant',
    content: [
      {
        type: 'text',
        text: cachedText,
        providerOptions,
      },
    ],
  },
  {
    role: 'user',
    content: 'Reply with "ok".',
  },
] satisfies ModelMessage[];

const cachedTextTool = tool({
  description: 'Return a stable cached text payload.',
  inputSchema: z.object({}),
  execute: async () => cachedText,
  toModelOutput: ({ output }) => ({
    type: 'text',
    value: output,
    providerOptions,
  }),
});

function hasCacheControl(body: unknown) {
  return JSON.stringify(body).includes('"cache_control"');
}

run(async () => {
  const controlCold = await generateText({
    model: anthropic('claude-opus-4-1'),
    messages: controlMessages,
    include: { requestBody: true },
  });

  console.log('=== control cold ===');
  console.log(controlCold.finalStep.providerMetadata?.anthropic?.usage);
  console.log('request has cache_control:', hasCacheControl(controlCold.request.body));

  const controlWarm = await generateText({
    model: anthropic('claude-opus-4-1'),
    messages: controlMessages,
    include: { requestBody: true },
  });

  console.log('=== control warm ===');
  console.log(controlWarm.finalStep.providerMetadata?.anthropic?.usage);
  console.log('request has cache_control:', hasCacheControl(controlWarm.request.body));

  const toolCold = await generateText({
    model: anthropic('claude-opus-4-1'),
    prompt: 'Call the cachedText tool, then reply with "ok".',
    tools: { cachedText: cachedTextTool },
    toolChoice: { type: 'tool', toolName: 'cachedText' },
    stopWhen: isStepCount(2),
    include: { requestBody: true },
  });

  console.log('=== tool result cold ===');
  console.log(toolCold.finalStep.providerMetadata?.anthropic?.usage);
  console.log('request has cache_control:', hasCacheControl(toolCold.finalStep.request.body));

  const toolWarm = await generateText({
    model: anthropic('claude-opus-4-1'),
    messages: [
      { role: 'user', content: 'Call the cachedText tool, then reply with "ok".' },
      ...toolCold.responseMessages,
      { role: 'user', content: 'Reply with "ok" again.' },
    ],
    tools: { cachedText: cachedTextTool },
    include: { requestBody: true },
  });

  console.log('=== tool result warm ===');
  console.log(toolWarm.finalStep.providerMetadata?.anthropic?.usage);
  console.log('request has cache_control:', hasCacheControl(toolWarm.request.body));
});

```
</details>

Observed behavior before the fix is that the last 2 tool-results will
show `request has cache_control: false`

## Checklist

- [x] All commits are signed (PRs with unsigned commits cannot be
merged)
- [x] Tests have been added / updated (for bug fixes / features)
- [ ] Documentation has been added / updated (for bug fixes / features)
- [x] A _patch_ changeset for relevant packages has been added (for bug
fixes / features - run `pnpm changeset` in the project root)
- [x] I have reviewed this pull request (self-review)

## Related Issues

fixes #15186
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@aayush-kapoor aayush-kapoor aayush-kapoor left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

tool.toModelOutput returned providerOptions silently dropped, Anthropic cache_control never reaches the API

2 participants

@Genmin @aayush-kapoor