Summary
This repo instruments boto3's bedrock-runtime service (model invoke/converse) via py/src/braintrust/integrations/bedrock_runtime/, but the separate bedrock-agent-runtime service — which drives Amazon Bedrock Agents' invoke_agent API (multi-step agent orchestration: action-group/tool calls, knowledge-base retrieval, and a returned execution trace) — is explicitly excluded, not just unimplemented.
py/src/braintrust/integrations/bedrock_runtime/patchers.py gates instrumentation with _is_bedrock_runtime_client, which only wraps a client when service_model.service_name == "bedrock-runtime". A boto3.client("bedrock-agent-runtime") client has a different service name and is passed straight through unwrapped, so calls to invoke_agent() (and invoke_flow() for Bedrock Flows) produce no spans at all.
This is a meaningful agent-run execution surface distinct from a single model invoke — it returns a streamed EventStream with agent reasoning/tool-use/observation chunks plus a final output, analogous to what's already traced for other agent frameworks in this repo (e.g. openai_agents, crewai, strands).
Braintrust docs status: unclear. https://www.braintrust.dev/docs/guides/tracing lists "AWS Bedrock" generically among supported providers but does not distinguish Bedrock Agents/invoke_agent from the base model-invoke surface, and no integration page documents agent-orchestration coverage.
What needs to be instrumented
| SDK surface |
Already traced? |
bedrock-agent-runtime client invoke_agent(agentId, agentAliasId, sessionId, inputText, ...) |
No |
invoke_flow(...) (Bedrock Flows) |
No |
Upstream sources
Local repo files inspected
py/src/braintrust/integrations/bedrock_runtime/integration.py — import_names = ("botocore",), min_version = "1.34.116", only patcher is BedrockClientCreatorPatcher
py/src/braintrust/integrations/bedrock_runtime/patchers.py — _is_bedrock_runtime_client checks service_model.service_name == "bedrock-runtime", explicitly excluding any other boto3 service name including bedrock-agent-runtime
py/src/braintrust/integrations/bedrock_runtime/tracing.py — wrappers exist only for converse, converse_stream, invoke_model, invoke_model_with_response_stream
- Repo-wide grep for
bedrock-agent, bedrock_agent, invoke_agent under py/src/braintrust/ — the only hit is an unrelated local test helper _invoke_agent in py/src/braintrust/integrations/langchain/test_deepagents.py (a LangChain/deepagents test fixture, unrelated to AWS)
README.md integrations table — no separate Bedrock Agents entry
Summary
This repo instruments boto3's
bedrock-runtimeservice (model invoke/converse) viapy/src/braintrust/integrations/bedrock_runtime/, but the separatebedrock-agent-runtimeservice — which drives Amazon Bedrock Agents'invoke_agentAPI (multi-step agent orchestration: action-group/tool calls, knowledge-base retrieval, and a returned executiontrace) — is explicitly excluded, not just unimplemented.py/src/braintrust/integrations/bedrock_runtime/patchers.pygates instrumentation with_is_bedrock_runtime_client, which only wraps a client whenservice_model.service_name == "bedrock-runtime". Aboto3.client("bedrock-agent-runtime")client has a different service name and is passed straight through unwrapped, so calls toinvoke_agent()(andinvoke_flow()for Bedrock Flows) produce no spans at all.This is a meaningful agent-run execution surface distinct from a single model invoke — it returns a streamed
EventStreamwith agent reasoning/tool-use/observation chunks plus a final output, analogous to what's already traced for other agent frameworks in this repo (e.g.openai_agents,crewai,strands).Braintrust docs status:
unclear. https://www.braintrust.dev/docs/guides/tracing lists "AWS Bedrock" generically among supported providers but does not distinguish Bedrock Agents/invoke_agentfrom the base model-invoke surface, and no integration page documents agent-orchestration coverage.What needs to be instrumented
bedrock-agent-runtimeclientinvoke_agent(agentId, agentAliasId, sessionId, inputText, ...)invoke_flow(...)(Bedrock Flows)Upstream sources
InvokeAgent: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent-runtime_InvokeAgent.htmlbedrock-agent-runtimeclient docs: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/bedrock-agent-runtime.htmlLocal repo files inspected
py/src/braintrust/integrations/bedrock_runtime/integration.py—import_names = ("botocore",),min_version = "1.34.116", only patcher isBedrockClientCreatorPatcherpy/src/braintrust/integrations/bedrock_runtime/patchers.py—_is_bedrock_runtime_clientchecksservice_model.service_name == "bedrock-runtime", explicitly excluding any other boto3 service name includingbedrock-agent-runtimepy/src/braintrust/integrations/bedrock_runtime/tracing.py— wrappers exist only forconverse,converse_stream,invoke_model,invoke_model_with_response_streambedrock-agent,bedrock_agent,invoke_agentunderpy/src/braintrust/— the only hit is an unrelated local test helper_invoke_agentinpy/src/braintrust/integrations/langchain/test_deepagents.py(a LangChain/deepagents test fixture, unrelated to AWS)README.mdintegrations table — no separate Bedrock Agents entry