Skip to content

Commit 5ef172c

Browse files
authored
feat(aws-sdk, llmobs): support Bedrock Converse and ConverseStream (#8079)
* feat(aws-sdk, llmobs): support Bedrock Converse and ConverseStream Adds APM span tagging and LLMObs enrichment for ConverseCommand and ConverseStreamCommand on the Bedrock runtime client. - tracing: extend operation whitelist - utils: Converse request/response parsers plus stream-event aggregator for text + toolUse content blocks - instrumentation: wrap `result.stream` as well as `result.body` so ConverseStream events flow through the streamed-chunk channel - llmobs: branch on Converse in setLLMObsTags, surface stop_reason, emit structured output messages with tool_calls - tests + VCR cassettes for system + tool path against Claude 3 Haiku Ref: MLOB-3509 * fix(aws-sdk, llmobs): surface unsupported Converse delta and tool-result variants Emit placeholder markers instead of dropping data silently: - buildToolResult: non-text/non-json content items (document, image, video, searchResult) now produce `[<type> content]` instead of ''. - buildConverseStreamGeneration: ContentBlockDelta events that are neither text nor toolUse (citation, image, reasoningContent, toolResult) now append `[Unsupported delta: <type>]` at their contentBlockIndex, so streamed responses using these variants no longer tag as empty output. Ref: MLOB-3509 * feat(aws-sdk, llmobs): emit tool_definitions for Bedrock Converse When a ConverseCommand or ConverseStreamCommand request includes a `toolConfig.tools` array, map each `toolSpec` to LLMObs' `{ name, description, schema }` shape and attach it to the span via the new `tagToolDefinitions` API (see parent PR #8082). - utils.js: new `extractConverseToolDefinitions` helper. - llmobs/plugins/bedrockruntime.js: call it on Converse requests. - spec: assert on the `toolDefinitions` round-trip. Brings Node parity with dd-trace-py's Bedrock Converse integration for the "Available Tools" UI section. Ref: MLOB-3509 * fix(aws-sdk, llmobs): null-guard Converse request extractors Malformed Converse payloads (e.g. `content: [null]`, `tools: [null]`) would throw in `extractMessagesFromConverseContent` and `extractConverseToolDefinitions`. Plugin subscriber exceptions trigger auto-disable via `this.configure(false)`, turning off Bedrock LLMObs for the rest of the process. Skip invalid entries instead of throwing, matching the defensive optional-chaining style already used by this file's InvokeModel extractors. Flagged by Codex adversarial review. * fix(aws-sdk, llmobs): match dd-trace-py unsupported tool-result marker Align `buildToolResult` fallback wording with dd-trace-py's `[Unsupported content type(s): <keys>]` format (see `dd-trace-py/ddtrace/llmobs/_integrations/utils.py:266`). * fix(aws-sdk, llmobs): drop unsupported-delta markers in Converse stream Dropping the `[Unsupported delta: <type>]` fallback: it spammed the output on every delta event (thinking-mode responses stream reasoningContent as dozens of small deltas) and had no parity in dd-trace-py. Match Python's silent-drop behavior for unhandled ContentBlockDelta variants; the non-stream `[Unsupported content type: ...]` markers already cover the debuggability need. * refactor(aws-sdk, llmobs): readability pass on Converse extractors - `buildToolCall`: extract `parseToolInput` helper so `args` is set once in a single expression (inputStr wins over input explicitly). - `buildToolResult`: extract `resolveToolResultItem` helper so the three cases (text / json / unsupported) are a flat if-chain instead of a nested ternary inside `.map`. - `extractConverseToolDefinitions`: rename `defs` -> `toolDefinitions`, `spec` -> `toolSpec` to match the AWS SDK field names. - `extractTextAndResponseReasonConverse`: rename the raw-response- handle `message` -> `outputMessage` so it no longer collides with the `messages` LLMObs array built right after. * refactor(aws-sdk, llmobs): split setLLMObsTags into per-API paths The original `setLLMObsTags` mixed InvokeModel and Converse flows, branching on `isConverse` / `isStream` in three places (request params, generation extraction, stop_reason emission). Split into two specialized methods so each path reads top-to-bottom: - `#tagConverseSpan` owns the Converse-only concerns: tool_definitions, stop_reason metadata, Converse request/response extractors. - `#tagInvokeModelSpan` owns the per-provider InvokeModel extractors. - `#tagCommon` handles the truly shared tagging (temperature, max_tokens, I/O, metrics) with no shape branching. To make `#tagCommon` shape-agnostic, `Generation` now auto-derives its structured `messages` field from `{message, role}` when the caller doesn't pass `messages` explicitly. The plugin reads `generation.messages` unconditionally; InvokeModel behavior is unchanged (same `[{content, role}]` wrapper, just produced at the class boundary instead of inline at the tag call site). Also: `stop_reason` moves out of the shared metadata path into `#tagConverseSpan`. It was leaking onto InvokeModel spans through the shared code; this restores pre-PR behavior for InvokeModel. * refactor(aws-sdk, llmobs): unify Converse stream and non-stream extractors Stream events describe the same content-block structure as the non- stream response, just chunked across start/delta events. Reassemble the stream chunks into a normalized `ContentBlock[]` shape and reuse the non-stream extractor via a shared `toOutputMessages` helper. - `buildConverseStreamGeneration`: replaces the parallel `textByIdx` / `toolByIdx` maps (and the dedicated `assembleStreamMessage` helper, now deleted) with a single `blocksByIdx` Map of reassembled blocks. One code path for content-block -> output-message regardless of stream vs non-stream. - `toOutputMessages`: new helper that wraps `extractMessagesFromConverseContent` with the "always emit at least one output message" fallback, shared by both paths. - `extractMessagesFromConverseContent`: early-return when nothing extracted; end-of-function double-check replaced with a plain `return [message]`. - `extractTextAndResponseReason` (InvokeModel Amazon path): replace the four explicit token fields with `...buildUsage(body.usage)`. Incidentally fixes a latent bug where `cacheRead/WriteInputTokenCount` were passed to a `Generation` constructor that only destructures `cacheRead/WriteTokens`, silently dropping those values. * fix(aws-sdk, llmobs): skip unsupported Converse stream delta variants The ContentBlockDelta union has 6 variants (text, toolUse, toolResult, reasoningContent, citation, image). Only text and toolUse.input were decoded; the rest are intended to be silently ignored per Python parity. The previous implementation leaked empty `{}` blocks into `blocksByIdx` when a delta arrived at a new contentBlockIndex with no matching handler (e.g. Claude 3.7 thinking mode emitting reasoningContent deltas). Downstream, `extractMessagesFromConverseContent` saw the empty block and emitted `[Unsupported content type: unknown]`. Fix: only write into `blocksByIdx` inside the branches that actually populate a block. Unhandled deltas now leave the Map untouched. Flagged by review on utils.js:647-651. * test(aws-sdk): silence unhandled rejection in Converse trace assertions If `send` (or stream iteration) rejects before the test reaches `await tracesPromise`, the trace-assertion promise stays pending and later settles with no observer, triggering an unhandled-rejection warning in some Node versions / CI configs. Attach a no-op `.catch()` to the original `tracesPromise` right after creation. This satisfies the "promise was handled" check without mutating the source — the subsequent `await tracesPromise` still sees the rejection and propagates it to the test runner. Flagged on review. * fix(aws-sdk, llmobs): only call tagToolDefinitions when tools are present `tagToolDefinitions` now logs a failure for non-array / empty input (see #8082). The Bedrock plugin previously called it unconditionally on every Converse request — `extractConverseToolDefinitions` returns `[]` when there's no `toolConfig`, which would now produce noisy `invalid_tool_definitions` logs on every tool-less Converse call. Gate the call on `length > 0`. * refactor(aws-sdk, llmobs): use generic isStream check and rename buildConverseStreamGeneration Pass isStream from setLLMObsTags to per-API tag methods instead of checking exact operation names, making the stream detection future-proof. Rename buildConverseStreamGeneration to extractTextAndResponseReasonConverseFromStream for consistency with the existing naming convention. * refactor(aws-sdk, llmobs): rename generation to textAndResponseReason Align local variable name with the extractTextAndResponseReason* function family it comes from. * refactor(aws-sdk, llmobs): clarify converse content-block member extraction - add getContentBlockType helper; unwraps $unknown tuple so forward-compat members surface their real type instead of the literal '$unknown' - extractMessagesFromConverseContent returns a single message or undefined instead of a 0-or-1-length array; callers adjusted * refactor(aws-sdk, llmobs): normalize converse stream tool input at reassembly Parse the accumulated tool-call argument string into the structured SDK input shape inside the stream extractor, so buildToolCall and the shared content-block extractor only ever see one block shape. Drops the dual input/inputStr branch from buildToolCall; inputStr is now private to the stream reassembly step. * test(aws-sdk, llmobs): cover converse content-block edge paths VCR specs only exercise happy paths. Add a fast channel-driven blackbox spec hitting the uncovered branches: toolResult rendering, unsupported content-block labeling, malformed stream tool JSON, and multi-delta text accumulation. * test(aws-sdk, llmobs): cover converse content blocks via VCR + unit Record a multi-turn converse-stream cassette (toolResult fed back, model streams a text answer) covering the recordable paths: stream text-delta accumulation and toolResult rendering. Keep the channel-driven unit spec only for the branches that cannot be recorded against live Bedrock: unsupported block types, unsupported tool-result items, and malformed streamed tool-use JSON. Allow real AWS credentials to override the dummy ones via the environment so the cassette can be regenerated. * test(aws-sdk, llmobs): drop noise comment above useEnv * fix(aws-sdk, llmobs): drop useless undefined in converse content extractor * test(aws-sdk, llmobs): cover converse content-block edge paths without tagger spies Drop the channel-driven converse spec that stubbed the tagger. Cover the unsupported content-block and tool-result-item branches via a hand-authored converse cassette through the real SDK path, and the malformed streamed tool-use JSON branch via a direct unit test of the exported stream extractor. Reuse getContentBlockType in resolveToolResultItem so unknown tool-result items surface their real type name instead of the $unknown wrapper.
1 parent c8eb110 commit 5ef172c

12 files changed

Lines changed: 719 additions & 32 deletions

packages/datadog-instrumentations/src/aws-sdk.js

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -233,13 +233,14 @@ function wrapSmithySend (send) {
233233
}
234234

235235
function handleCompletion (result, ctx, channels) {
236-
const iterator = result?.body?.[Symbol.asyncIterator]
236+
const streamable = result?.body ?? result?.stream
237+
const iterator = streamable?.[Symbol.asyncIterator]
237238
if (!iterator) {
238239
channels.complete.publish(ctx)
239240
return
240241
}
241242

242-
shimmer.wrap(result.body, Symbol.asyncIterator, function (asyncIterator) {
243+
shimmer.wrap(streamable, Symbol.asyncIterator, function (asyncIterator) {
243244
return function (...args) {
244245
const iterator = asyncIterator.apply(this, args)
245246
shimmer.wrap(iterator, 'next', function (next) {

packages/datadog-plugin-aws-sdk/src/services/bedrockruntime/tracing.js

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
const BaseAwsSdkPlugin = require('../../base')
44
const { parseModelId } = require('./utils')
55

6-
const enabledOperations = new Set(['invokeModel', 'invokeModelWithResponseStream'])
6+
const enabledOperations = new Set(['invokeModel', 'invokeModelWithResponseStream', 'converse', 'converseStream'])
77

88
class BedrockRuntime extends BaseAwsSdkPlugin {
99
static id = 'bedrockruntime'

packages/datadog-plugin-aws-sdk/src/services/bedrockruntime/utils.js

Lines changed: 218 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,7 @@ class Generation {
131131
outputTokens,
132132
cacheReadTokens,
133133
cacheWriteTokens,
134+
messages,
134135
} = {}) {
135136
// stringify message as it could be a single generated message as well as a list of embeddings
136137
this.message = typeof message === 'string' ? message : JSON.stringify(message) || ''
@@ -143,6 +144,7 @@ class Generation {
143144
cacheReadTokens,
144145
cacheWriteTokens,
145146
}
147+
this.messages = messages ?? [{ content: this.message, role: this.role }]
146148
}
147149
}
148150

@@ -401,10 +403,7 @@ function extractTextAndResponseReason (response, provider, modelName) {
401403
message: output.message?.content[0]?.text ?? 'Unsupported content type',
402404
finishReason: body.stopReason,
403405
role: output.message?.role,
404-
inputTokens: body.usage?.inputTokens,
405-
outputTokens: body.usage?.outputTokens,
406-
cacheReadInputTokenCount: body.usage?.cacheReadInputTokenCount,
407-
cacheWriteInputTokenCount: body.usage?.cacheWriteInputTokenCount,
406+
...buildUsage(body.usage),
408407
})
409408
}
410409
break
@@ -476,12 +475,227 @@ function extractTextAndResponseReason (response, provider, modelName) {
476475
return new Generation()
477476
}
478477

478+
/**
479+
* Convert a Converse content-block array to an LLMObs message array.
480+
*
481+
* @param {string} role
482+
* @param {Array<object>} contentBlocks
483+
* @returns {{ content?: string, role: string, toolCalls?: Array, toolResults?: Array } | undefined}
484+
*/
485+
function extractMessagesFromConverseContent (role, contentBlocks) {
486+
let content = ''
487+
const toolCalls = []
488+
const toolResults = []
489+
490+
for (const block of contentBlocks || []) {
491+
if (block == null || typeof block !== 'object') continue
492+
if (typeof block.text === 'string') {
493+
content += block.text
494+
} else if (block.toolUse) {
495+
toolCalls.push(buildToolCall(block.toolUse))
496+
} else if (block.toolResult) {
497+
toolResults.push(buildToolResult(block.toolResult))
498+
} else {
499+
content += `[Unsupported content type: ${getContentBlockType(block)}]`
500+
}
501+
}
502+
503+
if (!content && toolCalls.length === 0 && toolResults.length === 0) return
504+
505+
const message = { role }
506+
if (content) message.content = content
507+
if (toolCalls.length > 0) message.toolCalls = toolCalls
508+
if (toolResults.length > 0) message.toolResults = toolResults
509+
return message
510+
}
511+
512+
/**
513+
* Resolve a Converse `ContentBlock`'s member type. The block is a key-presence
514+
* tagged union (no `type` discriminator), so the active member is its sole own
515+
* key. For forward-compat `$unknown` members the real type is the first element
516+
* of the `[name, value]` tuple.
517+
*
518+
* @param {object} block
519+
* @returns {string}
520+
*/
521+
function getContentBlockType (block) {
522+
const key = Object.keys(block)[0]
523+
if (key === '$unknown') return block.$unknown?.[0] ?? 'unknown'
524+
return key ?? 'unknown'
525+
}
526+
527+
// Always emit at least one output message so downstream tagging has a role to attach to.
528+
function toOutputMessages (role, contentBlocks) {
529+
const message = extractMessagesFromConverseContent(role, contentBlocks)
530+
return message ? [message] : [{ role, content: '' }]
531+
}
532+
533+
function buildToolCall ({ name, input, toolUseId }) {
534+
return { name: name ?? '', arguments: input ?? {}, toolId: toolUseId ?? '', type: 'toolUse' }
535+
}
536+
537+
function parseToolInput (inputStr) {
538+
try {
539+
return JSON.parse(inputStr)
540+
} catch {
541+
log.warn('Failed to parse Converse stream toolUse.input JSON; emitting empty arguments')
542+
return {}
543+
}
544+
}
545+
546+
function buildToolResult ({ toolUseId, content }) {
547+
const result = (content || []).map(resolveToolResultItem).join('')
548+
return { name: '', result, toolId: toolUseId ?? '', type: 'tool_result' }
549+
}
550+
551+
function resolveToolResultItem (item) {
552+
if (typeof item.text === 'string') return item.text
553+
if (item.json != null) return JSON.stringify(item.json)
554+
return `[Unsupported content type(s): ${getContentBlockType(item)}]`
555+
}
556+
557+
function buildUsage (usage = {}) {
558+
return {
559+
inputTokens: usage.inputTokens,
560+
outputTokens: usage.outputTokens,
561+
cacheReadTokens: usage.cacheReadInputTokens ?? usage.cacheReadInputTokenCount,
562+
cacheWriteTokens: usage.cacheWriteInputTokens ?? usage.cacheWriteInputTokenCount,
563+
}
564+
}
565+
566+
/**
567+
* Extract tool definitions from a Converse request's `toolConfig.tools`,
568+
* mapping Bedrock's `toolSpec` shape to LLMObs `ToolDefinition` shape.
569+
*
570+
* @param {object} params - Converse request params with optional `toolConfig.tools[].toolSpec`.
571+
* @returns {Array<{ name: string, description: string, schema: object }>}
572+
*/
573+
function extractConverseToolDefinitions (params) {
574+
const toolDefinitions = []
575+
for (const tool of params.toolConfig?.tools || []) {
576+
const toolSpec = tool?.toolSpec
577+
if (!toolSpec?.name) continue
578+
toolDefinitions.push({
579+
name: toolSpec.name,
580+
description: toolSpec.description ?? '',
581+
schema: toolSpec.inputSchema ?? {},
582+
})
583+
}
584+
return toolDefinitions
585+
}
586+
587+
/**
588+
* Extract request metadata + rendered input messages from a Converse /
589+
* ConverseStream request.
590+
*
591+
* @param {{ modelId?: string, messages?: Array, system?: Array, inferenceConfig?: object, toolConfig?: object }} params
592+
* @returns {RequestParams}
593+
*/
594+
function extractRequestParamsConverse (params) {
595+
const prompt = []
596+
for (const block of params.system || []) {
597+
if (typeof block?.text === 'string') prompt.push({ content: block.text, role: 'system' })
598+
}
599+
for (const msg of params.messages || []) {
600+
if (msg == null || typeof msg !== 'object') continue
601+
const message = extractMessagesFromConverseContent(msg.role || 'user', msg.content)
602+
if (message) prompt.push(message)
603+
}
604+
605+
const { temperature, topP, maxTokens, stopSequences } = params.inferenceConfig || {}
606+
return new RequestParams({ prompt, temperature, topP, maxTokens, stopSequences })
607+
}
608+
609+
/**
610+
* Extract output messages + usage from a non-stream Converse response.
611+
*
612+
* @param {{ output?: { message?: { role?: string, content?: Array } }, stopReason?: string, usage?: object }} response
613+
* @returns {Generation}
614+
*/
615+
function extractTextAndResponseReasonConverse (response) {
616+
const outputMessage = response?.output?.message
617+
const role = outputMessage?.role || 'assistant'
618+
619+
return new Generation({
620+
role,
621+
finishReason: response?.stopReason || '',
622+
...buildUsage(response?.usage),
623+
messages: toOutputMessages(role, outputMessage?.content),
624+
})
625+
}
626+
627+
/**
628+
* Aggregate Converse stream events into a single output message + usage.
629+
* One messageStart / messageStop pair per response, so one message out.
630+
*
631+
* Stream events describe the same content-block structure as the non-stream
632+
* response, spread across start/delta chunks. We reassemble those chunks
633+
* into a normalized content-block array and reuse the non-stream extractor.
634+
*
635+
* @param {Array<object>} chunks - Ordered ConverseStreamOutput events.
636+
* @returns {Generation}
637+
*/
638+
function extractTextAndResponseReasonConverseFromStream (chunks) {
639+
let role = 'assistant'
640+
let stopReason = ''
641+
let usage = {}
642+
const blocksByIdx = new Map()
643+
644+
for (const chunk of chunks || []) {
645+
if (chunk.messageStart?.role) {
646+
role = chunk.messageStart.role
647+
} else if (chunk.messageStop?.stopReason) {
648+
stopReason = chunk.messageStop.stopReason
649+
} else if (chunk.metadata?.usage) {
650+
usage = chunk.metadata.usage
651+
} else if (chunk.contentBlockStart?.start?.toolUse) {
652+
const { contentBlockIndex, start: { toolUse } } = chunk.contentBlockStart
653+
blocksByIdx.set(contentBlockIndex, {
654+
toolUse: { toolUseId: toolUse.toolUseId, name: toolUse.name, inputStr: '' },
655+
})
656+
} else if (chunk.contentBlockDelta) {
657+
const { contentBlockIndex, delta } = chunk.contentBlockDelta
658+
if (typeof delta?.text === 'string') {
659+
const block = blocksByIdx.get(contentBlockIndex) ?? {}
660+
block.text = (block.text ?? '') + delta.text
661+
blocksByIdx.set(contentBlockIndex, block)
662+
} else if (typeof delta?.toolUse?.input === 'string') {
663+
const block = blocksByIdx.get(contentBlockIndex) ?? { toolUse: { inputStr: '' } }
664+
block.toolUse ??= { inputStr: '' }
665+
block.toolUse.inputStr += delta.toolUse.input
666+
blocksByIdx.set(contentBlockIndex, block)
667+
}
668+
}
669+
}
670+
671+
const contentBlocks = [...blocksByIdx.keys()].sort((a, b) => a - b).map(i => {
672+
const block = blocksByIdx.get(i)
673+
if (block.toolUse) {
674+
const { toolUseId, name, inputStr } = block.toolUse
675+
block.toolUse = { toolUseId, name, input: parseToolInput(inputStr) }
676+
}
677+
return block
678+
})
679+
680+
return new Generation({
681+
role,
682+
finishReason: stopReason,
683+
...buildUsage(usage),
684+
messages: toOutputMessages(role, contentBlocks),
685+
})
686+
}
687+
479688
module.exports = {
480689
Generation,
481690
RequestParams,
482691
extractTextAndResponseReasonFromStream,
483692
parseModelId,
484693
extractRequestParams,
485694
extractTextAndResponseReason,
695+
extractMessagesFromConverseContent,
696+
extractConverseToolDefinitions,
697+
extractRequestParamsConverse,
698+
extractTextAndResponseReasonConverse,
699+
extractTextAndResponseReasonConverseFromStream,
486700
PROVIDER,
487701
}

packages/datadog-plugin-aws-sdk/test/bedrockruntime.spec.js

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ const { describe, it, before, after } = require('mocha')
55

66
const agent = require('../../dd-trace/test/plugins/agent')
77
const { setup, withAwsSdkVersions } = require('./spec_helpers')
8-
const { models } = require('./fixtures/bedrockruntime')
8+
const { models, converseRequest } = require('./fixtures/bedrockruntime')
99
const serviceName = 'bedrock-service-name-test'
1010

1111
describe('Plugin', () => {
@@ -97,6 +97,46 @@ describe('Plugin', () => {
9797
await tracesPromise
9898
})
9999
})
100+
101+
it('should converse', async function () {
102+
if (typeof AWS.ConverseCommand !== 'function') return this.skip()
103+
const command = new AWS.ConverseCommand({ modelId: converseRequest.modelId, ...converseRequest.request })
104+
105+
const tracesPromise = agent.assertFirstTraceSpan({
106+
meta: {
107+
'aws.operation': 'converse',
108+
'aws.bedrock.request.model': converseRequest.modelId.split('.')[1],
109+
'aws.bedrock.request.model_provider': converseRequest.provider.toLowerCase(),
110+
},
111+
})
112+
tracesPromise.catch(() => {}) // silence unhandled rejection if `send` throws first
113+
114+
await bedrockRuntimeClient.send(command)
115+
await tracesPromise
116+
})
117+
118+
it('should converse-stream', async function () {
119+
if (typeof AWS.ConverseStreamCommand !== 'function') return this.skip()
120+
const command = new AWS.ConverseStreamCommand({
121+
modelId: converseRequest.modelId,
122+
...converseRequest.request,
123+
})
124+
125+
const tracesPromise = agent.assertFirstTraceSpan({
126+
meta: {
127+
'aws.operation': 'converseStream',
128+
'aws.bedrock.request.model': converseRequest.modelId.split('.')[1],
129+
'aws.bedrock.request.model_provider': converseRequest.provider.toLowerCase(),
130+
},
131+
})
132+
tracesPromise.catch(() => {}) // silence unhandled rejection if stream iteration throws first
133+
134+
const result = await bedrockRuntimeClient.send(command)
135+
for await (const _event of result.stream) { // eslint-disable-line no-unused-vars
136+
// drain
137+
}
138+
await tracesPromise
139+
})
100140
})
101141
})
102142
})
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
'use strict'
2+
3+
const assert = require('node:assert/strict')
4+
5+
const { describe, it } = require('mocha')
6+
7+
const {
8+
extractTextAndResponseReasonConverseFromStream,
9+
} = require('../src/services/bedrockruntime/utils')
10+
11+
describe('bedrockruntime converse stream extractor', () => {
12+
it('emits empty tool-call arguments when the streamed tool-use input is malformed JSON', () => {
13+
const generation = extractTextAndResponseReasonConverseFromStream([
14+
{ messageStart: { role: 'assistant' } },
15+
{ contentBlockStart: { contentBlockIndex: 0, start: { toolUse: { toolUseId: 't-1', name: 'get_weather' } } } },
16+
{ contentBlockDelta: { contentBlockIndex: 0, delta: { toolUse: { input: 'not valid json{' } } } },
17+
{ messageStop: { stopReason: 'tool_use' } },
18+
])
19+
20+
assert.deepStrictEqual(generation.messages, [{
21+
role: 'assistant',
22+
toolCalls: [{ name: 'get_weather', arguments: {}, toolId: 't-1', type: 'toolUse' }],
23+
}])
24+
})
25+
})

0 commit comments

Comments
 (0)