anthropic[patch]: Handle non text llm tokens

[bracesproul]

Closes #6045

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchainjs-api-refs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 23, 2024 7:04pm
langchainjs-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 23, 2024 7:04pm

[snlamm]

@bracesproul I think instead/in addition we need handle type: 'input_json_delta'?

Here's a snippet of the chunks that are generated when I run a prompt using tools - the actual text is in the input_json_delta:

[
  {
    index: 0,
    type: 'tool_use',
    id: 'toolu_01DfwKsAZ97rf3zLJmVd6cyU',
    name: 'extract',
    input: ''
  }
]
[ { index: 0, input: '', type: 'input_json_delta' } ]
[ { index: 0, input: '{"choi', type: 'input_json_delta' } ]
[ { index: 0, input: 'ces": [{"', type: 'input_json_delta' } ]
[ { index: 0, input: 'type": ', type: 'input_json_delta' } ]
[ { index: 0, input: '"knowle', type: 'input_json_delta' } ]
[ { index: 0, input: 'dge', type: 'input_json_delta' } ]
[ { index: 0, input: 'Overview", ', type: 'input_json_delta' } ]
[ { index: 0, input: '"description', type: 'input_json_delta' } ]
[ { index: 0, input: '": "', type: 'input_json_delta' } ]
[ { index: 0, input: 'Identify', type: 'input_json_delta' } ]
...

[bracesproul]

@bracesproul I think instead/in addition we need handle type: 'input_json_delta'?

Here's a snippet of the chunks that are generated when I run a prompt using tools - the actual text is in the input_json_delta:

[
  {
    index: 0,
    type: 'tool_use',
    id: 'toolu_01DfwKsAZ97rf3zLJmVd6cyU',
    name: 'extract',
    input: ''
  }
]
[ { index: 0, input: '', type: 'input_json_delta' } ]
[ { index: 0, input: '{"choi', type: 'input_json_delta' } ]
[ { index: 0, input: 'ces": [{"', type: 'input_json_delta' } ]
[ { index: 0, input: 'type": ', type: 'input_json_delta' } ]
[ { index: 0, input: '"knowle', type: 'input_json_delta' } ]
[ { index: 0, input: 'dge', type: 'input_json_delta' } ]
[ { index: 0, input: 'Overview", ', type: 'input_json_delta' } ]
[ { index: 0, input: '"description', type: 'input_json_delta' } ]
[ { index: 0, input: '": "', type: 'input_json_delta' } ]
[ { index: 0, input: 'Identify', type: 'input_json_delta' } ]
...

Updated to just check if input exists.

[snlamm]

@bracesproul after trying it out, it looks like the current implementation accidentally breaks on the first message_start event. I've opened a PR to fix here if that's helpful.