Commit 5ef172c
authored
feat(aws-sdk, llmobs): support Bedrock Converse and ConverseStream (#8079)
* feat(aws-sdk, llmobs): support Bedrock Converse and ConverseStream
Adds APM span tagging and LLMObs enrichment for ConverseCommand and
ConverseStreamCommand on the Bedrock runtime client.
- tracing: extend operation whitelist
- utils: Converse request/response parsers plus stream-event aggregator
for text + toolUse content blocks
- instrumentation: wrap `result.stream` as well as `result.body` so
ConverseStream events flow through the streamed-chunk channel
- llmobs: branch on Converse in setLLMObsTags, surface stop_reason,
emit structured output messages with tool_calls
- tests + VCR cassettes for system + tool path against Claude 3 Haiku
Ref: MLOB-3509
* fix(aws-sdk, llmobs): surface unsupported Converse delta and tool-result variants
Emit placeholder markers instead of dropping data silently:
- buildToolResult: non-text/non-json content items (document, image,
video, searchResult) now produce `[<type> content]` instead of ''.
- buildConverseStreamGeneration: ContentBlockDelta events that are
neither text nor toolUse (citation, image, reasoningContent,
toolResult) now append `[Unsupported delta: <type>]` at their
contentBlockIndex, so streamed responses using these variants no
longer tag as empty output.
Ref: MLOB-3509
* feat(aws-sdk, llmobs): emit tool_definitions for Bedrock Converse
When a ConverseCommand or ConverseStreamCommand request includes a
`toolConfig.tools` array, map each `toolSpec` to LLMObs'
`{ name, description, schema }` shape and attach it to the span via
the new `tagToolDefinitions` API (see parent PR #8082).
- utils.js: new `extractConverseToolDefinitions` helper.
- llmobs/plugins/bedrockruntime.js: call it on Converse requests.
- spec: assert on the `toolDefinitions` round-trip.
Brings Node parity with dd-trace-py's Bedrock Converse integration
for the "Available Tools" UI section.
Ref: MLOB-3509
* fix(aws-sdk, llmobs): null-guard Converse request extractors
Malformed Converse payloads (e.g. `content: [null]`, `tools: [null]`)
would throw in `extractMessagesFromConverseContent` and
`extractConverseToolDefinitions`. Plugin subscriber exceptions trigger
auto-disable via `this.configure(false)`, turning off Bedrock LLMObs
for the rest of the process.
Skip invalid entries instead of throwing, matching the defensive
optional-chaining style already used by this file's InvokeModel
extractors.
Flagged by Codex adversarial review.
* fix(aws-sdk, llmobs): match dd-trace-py unsupported tool-result marker
Align `buildToolResult` fallback wording with dd-trace-py's
`[Unsupported content type(s): <keys>]` format (see
`dd-trace-py/ddtrace/llmobs/_integrations/utils.py:266`).
* fix(aws-sdk, llmobs): drop unsupported-delta markers in Converse stream
Dropping the `[Unsupported delta: <type>]` fallback: it spammed the
output on every delta event (thinking-mode responses stream
reasoningContent as dozens of small deltas) and had no parity in
dd-trace-py. Match Python's silent-drop behavior for unhandled
ContentBlockDelta variants; the non-stream `[Unsupported content
type: ...]` markers already cover the debuggability need.
* refactor(aws-sdk, llmobs): readability pass on Converse extractors
- `buildToolCall`: extract `parseToolInput` helper so `args` is set
once in a single expression (inputStr wins over input explicitly).
- `buildToolResult`: extract `resolveToolResultItem` helper so the
three cases (text / json / unsupported) are a flat if-chain instead
of a nested ternary inside `.map`.
- `extractConverseToolDefinitions`: rename `defs` -> `toolDefinitions`,
`spec` -> `toolSpec` to match the AWS SDK field names.
- `extractTextAndResponseReasonConverse`: rename the raw-response-
handle `message` -> `outputMessage` so it no longer collides with
the `messages` LLMObs array built right after.
* refactor(aws-sdk, llmobs): split setLLMObsTags into per-API paths
The original `setLLMObsTags` mixed InvokeModel and Converse flows,
branching on `isConverse` / `isStream` in three places (request
params, generation extraction, stop_reason emission). Split into
two specialized methods so each path reads top-to-bottom:
- `#tagConverseSpan` owns the Converse-only concerns:
tool_definitions, stop_reason metadata, Converse request/response
extractors.
- `#tagInvokeModelSpan` owns the per-provider InvokeModel extractors.
- `#tagCommon` handles the truly shared tagging (temperature,
max_tokens, I/O, metrics) with no shape branching.
To make `#tagCommon` shape-agnostic, `Generation` now auto-derives
its structured `messages` field from `{message, role}` when the
caller doesn't pass `messages` explicitly. The plugin reads
`generation.messages` unconditionally; InvokeModel behavior is
unchanged (same `[{content, role}]` wrapper, just produced at the
class boundary instead of inline at the tag call site).
Also: `stop_reason` moves out of the shared metadata path into
`#tagConverseSpan`. It was leaking onto InvokeModel spans through
the shared code; this restores pre-PR behavior for InvokeModel.
* refactor(aws-sdk, llmobs): unify Converse stream and non-stream extractors
Stream events describe the same content-block structure as the non-
stream response, just chunked across start/delta events. Reassemble
the stream chunks into a normalized `ContentBlock[]` shape and reuse
the non-stream extractor via a shared `toOutputMessages` helper.
- `buildConverseStreamGeneration`: replaces the parallel
`textByIdx` / `toolByIdx` maps (and the dedicated
`assembleStreamMessage` helper, now deleted) with a single
`blocksByIdx` Map of reassembled blocks. One code path for
content-block -> output-message regardless of stream vs non-stream.
- `toOutputMessages`: new helper that wraps
`extractMessagesFromConverseContent` with the "always emit at least
one output message" fallback, shared by both paths.
- `extractMessagesFromConverseContent`: early-return when nothing
extracted; end-of-function double-check replaced with a plain
`return [message]`.
- `extractTextAndResponseReason` (InvokeModel Amazon path): replace
the four explicit token fields with `...buildUsage(body.usage)`.
Incidentally fixes a latent bug where `cacheRead/WriteInputTokenCount`
were passed to a `Generation` constructor that only destructures
`cacheRead/WriteTokens`, silently dropping those values.
* fix(aws-sdk, llmobs): skip unsupported Converse stream delta variants
The ContentBlockDelta union has 6 variants (text, toolUse, toolResult,
reasoningContent, citation, image). Only text and toolUse.input were
decoded; the rest are intended to be silently ignored per Python
parity.
The previous implementation leaked empty `{}` blocks into
`blocksByIdx` when a delta arrived at a new contentBlockIndex with no
matching handler (e.g. Claude 3.7 thinking mode emitting
reasoningContent deltas). Downstream,
`extractMessagesFromConverseContent` saw the empty block and emitted
`[Unsupported content type: unknown]`.
Fix: only write into `blocksByIdx` inside the branches that actually
populate a block. Unhandled deltas now leave the Map untouched.
Flagged by review on utils.js:647-651.
* test(aws-sdk): silence unhandled rejection in Converse trace assertions
If `send` (or stream iteration) rejects before the test reaches
`await tracesPromise`, the trace-assertion promise stays pending and
later settles with no observer, triggering an unhandled-rejection
warning in some Node versions / CI configs.
Attach a no-op `.catch()` to the original `tracesPromise` right after
creation. This satisfies the "promise was handled" check without
mutating the source — the subsequent `await tracesPromise` still
sees the rejection and propagates it to the test runner.
Flagged on review.
* fix(aws-sdk, llmobs): only call tagToolDefinitions when tools are present
`tagToolDefinitions` now logs a failure for non-array / empty input
(see #8082). The Bedrock plugin previously called it unconditionally
on every Converse request — `extractConverseToolDefinitions` returns
`[]` when there's no `toolConfig`, which would now produce noisy
`invalid_tool_definitions` logs on every tool-less Converse call.
Gate the call on `length > 0`.
* refactor(aws-sdk, llmobs): use generic isStream check and rename buildConverseStreamGeneration
Pass isStream from setLLMObsTags to per-API tag methods instead of
checking exact operation names, making the stream detection future-proof.
Rename buildConverseStreamGeneration to
extractTextAndResponseReasonConverseFromStream for consistency with
the existing naming convention.
* refactor(aws-sdk, llmobs): rename generation to textAndResponseReason
Align local variable name with the extractTextAndResponseReason*
function family it comes from.
* refactor(aws-sdk, llmobs): clarify converse content-block member extraction
- add getContentBlockType helper; unwraps $unknown tuple so forward-compat
members surface their real type instead of the literal '$unknown'
- extractMessagesFromConverseContent returns a single message or undefined
instead of a 0-or-1-length array; callers adjusted
* refactor(aws-sdk, llmobs): normalize converse stream tool input at reassembly
Parse the accumulated tool-call argument string into the structured SDK
input shape inside the stream extractor, so buildToolCall and the shared
content-block extractor only ever see one block shape. Drops the dual
input/inputStr branch from buildToolCall; inputStr is now private to the
stream reassembly step.
* test(aws-sdk, llmobs): cover converse content-block edge paths
VCR specs only exercise happy paths. Add a fast channel-driven blackbox
spec hitting the uncovered branches: toolResult rendering, unsupported
content-block labeling, malformed stream tool JSON, and multi-delta text
accumulation.
* test(aws-sdk, llmobs): cover converse content blocks via VCR + unit
Record a multi-turn converse-stream cassette (toolResult fed back, model
streams a text answer) covering the recordable paths: stream text-delta
accumulation and toolResult rendering. Keep the channel-driven unit spec
only for the branches that cannot be recorded against live Bedrock:
unsupported block types, unsupported tool-result items, and malformed
streamed tool-use JSON. Allow real AWS credentials to override the dummy
ones via the environment so the cassette can be regenerated.
* test(aws-sdk, llmobs): drop noise comment above useEnv
* fix(aws-sdk, llmobs): drop useless undefined in converse content extractor
* test(aws-sdk, llmobs): cover converse content-block edge paths without tagger spies
Drop the channel-driven converse spec that stubbed the tagger. Cover the
unsupported content-block and tool-result-item branches via a hand-authored
converse cassette through the real SDK path, and the malformed streamed
tool-use JSON branch via a direct unit test of the exported stream extractor.
Reuse getContentBlockType in resolveToolResultItem so unknown tool-result
items surface their real type name instead of the $unknown wrapper.1 parent c8eb110 commit 5ef172c
12 files changed
Lines changed: 719 additions & 32 deletions
File tree
- packages
- datadog-instrumentations/src
- datadog-plugin-aws-sdk
- src/services/bedrockruntime
- test
- fixtures
- dd-trace
- src/llmobs/plugins
- test/llmobs
- cassettes/bedrock-runtime
- plugins/aws-sdk
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
233 | 233 | | |
234 | 234 | | |
235 | 235 | | |
236 | | - | |
| 236 | + | |
| 237 | + | |
237 | 238 | | |
238 | 239 | | |
239 | 240 | | |
240 | 241 | | |
241 | 242 | | |
242 | | - | |
| 243 | + | |
243 | 244 | | |
244 | 245 | | |
245 | 246 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| |||
Lines changed: 218 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
131 | 131 | | |
132 | 132 | | |
133 | 133 | | |
| 134 | + | |
134 | 135 | | |
135 | 136 | | |
136 | 137 | | |
| |||
143 | 144 | | |
144 | 145 | | |
145 | 146 | | |
| 147 | + | |
146 | 148 | | |
147 | 149 | | |
148 | 150 | | |
| |||
401 | 403 | | |
402 | 404 | | |
403 | 405 | | |
404 | | - | |
405 | | - | |
406 | | - | |
407 | | - | |
| 406 | + | |
408 | 407 | | |
409 | 408 | | |
410 | 409 | | |
| |||
476 | 475 | | |
477 | 476 | | |
478 | 477 | | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
| 594 | + | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
| 621 | + | |
| 622 | + | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
479 | 688 | | |
480 | 689 | | |
481 | 690 | | |
482 | 691 | | |
483 | 692 | | |
484 | 693 | | |
485 | 694 | | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
486 | 700 | | |
487 | 701 | | |
Lines changed: 41 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
| 8 | + | |
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| |||
97 | 97 | | |
98 | 98 | | |
99 | 99 | | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
100 | 140 | | |
101 | 141 | | |
102 | 142 | | |
| |||
Lines changed: 25 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
0 commit comments