Skip to content

Add AWS Bedrock provider (Converse + ConverseStream)#461

Open
ricardosllm wants to merge 1 commit into
editor-code-assistant:masterfrom
ricardosllm:bedrock-provider
Open

Add AWS Bedrock provider (Converse + ConverseStream)#461
ricardosllm wants to merge 1 commit into
editor-code-assistant:masterfrom
ricardosllm:bedrock-provider

Conversation

@ricardosllm
Copy link
Copy Markdown
Contributor

@ricardosllm ricardosllm commented May 14, 2026

Closes #254.

Adds a bedrock provider/API type using AWS Bedrock's native Converse and ConverseStream APIs.

Why

Bedrock's OpenAI-compatible endpoint only serves openai.* models — Claude (and other) inference profiles return 404 there, so the existing OpenAI-compat custom-provider path can't reach them. This adds a dedicated handler. (Also supersedes the partial workaround discussed in #329.)

Scope

  • Converse (non-streaming) and ConverseStream (streaming) — includes a binary vnd.amazon.eventstream frame decoder
  • Bearer-token auth (Authorization: Bearer <token>) — sidesteps SigV4; pairs with AWS_BEARER_TOKEN_BEDROCK
  • Tool callingtoolConfig / toolSpec / toolUse / toolResult blocks
  • ReasoningreasoningContent blocks; reasoning_config via additionalModelRequestFields when reason?
  • Region is part of the runtime URL; model ids / inference-profile ids are user-configured
  • extraPayload is filtered to Converse-valid top-level keys (verified against the botocore service model) so variant payloads aimed at other APIs don't 400
  • Retries on transient statuses reuse the existing error classification; modeled stream error frames terminate the turn cleanly (no spurious premature-stop)

SigV4 auth and a model-catalog endpoint are intentionally out of scope for this first cut.

Config

{
  "providers": {
    "bedrock": {
      "api": "bedrock",
      "url": "https://bedrock-runtime.us-east-1.amazonaws.com",
      "key": "${env:AWS_BEARER_TOKEN_BEDROCK}",
      "models": {
        "us.anthropic.claude-sonnet-4-5-20250929-v1:0": {}
      }
    }
  }
}

Also reachable via the /login flow (provider bedrock).

How to verify / test

Automatedbb test (suite green, 523 tests, 2881 assertions). Bedrock-specific: clojure -M:test --focus eca.llm-providers.bedrock-test covers the binary event-stream decoder (crafted frames, truncated/malformed), message normalization, extraPayload filtering, request/response shapes, both tool-call loops, streaming callbacks, and error paths.

Manual against live AWS:

  1. Generate a Bedrock API key in the AWS console (Bedrock → API keys) and export it: export AWS_BEARER_TOKEN_BEDROCK=...
  2. Build: bb debug-cli./eca
  3. Add the config above (adjust region in the URL and the model id to one enabled in your account), or use /loginbedrock.
  4. Point your editor/client at the built ./eca, select a bedrock/... model, and run a chat. Verify:
    • streaming text renders incrementally
    • a prompt that triggers a tool call round-trips (tool runs, model continues)
    • reasoning content shows when using a reasoning-capable model

This branch was integrated and smoke-tested live (non-streaming text, ConverseStream streaming, multi-turn tool-call round-trip) by a downstream consumer that embeds ECA as a library.


  • I added an entry in the changelog under the unreleased section.
  • This is not an AI slop.

🤖 Generated with Claude Code

Bedrock's OpenAI-compatible endpoint only serves openai.* models; Claude
and other inference profiles are reachable only through the native
Converse API. This adds a dedicated `bedrock` API type implementing both
Converse (non-streaming) and ConverseStream (binary event-stream framing),
with bearer-token auth, tool calling and reasoning.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ricardosllm
Copy link
Copy Markdown
Contributor Author

I wrote this with Claude Opus 4.7 and reviewed as well with OpenAI GPT 5.5

It works for my usecase, here's what I tested:

Configuration. Bedrock configured as a standard provider entry:

  {:providers {"bedrock" {:api    "bedrock"
                          :url    "https://bedrock-runtime.ap-southeast-1.amazonaws.com"
                          :key    "<bearer token>"
                          :models {"global.anthropic.claude-sonnet-4-6" {}}}}}
  • Region ap-southeast-1, runtime URL as above.
  • Bearer-token auth (Authorization: Bearer) via config :key.
  • Model: global.anthropic.claude-sonnet-4-6 (also configured global.anthropic.claude-opus-4-7).
  • Driven through eca.llm-api/sync-or-async-prompt! with :provider "bedrock", the per-call :config above, and :model-capabilities — no
    :models model-config entry needed; the generic branch in llm-api/prompt! keyed on (or model-config model-capabilities) handles it.

Proven working — live against AWS Bedrock:

  • Non-streaming Converse — text response, usage parsing.
  • ConverseStream — token-by-token streaming, messageStop/end_turn, finish + usage callbacks.
  • Tool calling — multi-turn round-trip: model emits toolUse, on-tools-called returns {:new-messages :tools}, ECA re-issues, model incorporates the tool result into the final answer. The on-tools-called contract ([{:id :full-name :arguments}] in, {:new-messages :tools} out) round-trips with no adapter glue.
  • provider->api-handler's :else branch correctly routes :api "bedrock" to the native handler.
  • Streaming chat and agentic tool workstreams confirmed end-to-end.

Not covered:

  • Reasoning/thinking path (reasoningContent / redactedContent)
  • Image input (supports-image?)
  • :stream false via extra-payload (default streaming + the non-stream handler path were tested)
  • Retry/error classification under real 429/503 from AWS
  • Mid-stream cancellation
  • Parallel tool calls (one tool call per turn tested)
  • Long multi-turn conversations
  • Non-Claude Bedrock models / inference profiles beyond the global Anthropic ones

If you think this is in the right direction I can try to test what was not covered yet.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class support for AWS Bedrock’s native runtime APIs (Converse + ConverseStream) as a new bedrock API/provider in ECA, enabling access to models/inference profiles that aren’t reachable via Bedrock’s OpenAI-compatible endpoint.

Changes:

  • Implement eca.llm-providers.bedrock including message normalization, tool-calling loops, and an AWS vnd.amazon.eventstream frame decoder for streaming.
  • Wire the new bedrock API into provider selection and /login provider configuration metadata.
  • Document configuration/schema updates and add a dedicated Bedrock test suite.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/eca/llm_providers/bedrock.clj New Bedrock provider implementation (Converse/ConverseStream + event-stream decoding + tools/reasoning support).
test/eca/llm_providers/bedrock_test.clj Comprehensive tests for decoder, normalization, streaming/tool loops, and error handling.
src/eca/llm_api.clj Register bedrock API handler for provider configs.
src/eca/features/providers.clj Add “AWS Bedrock” provider label, login fields, and default provider config.
docs/config/models.md Add Bedrock configuration documentation and example.
docs/config.json Extend schema enum to allow "bedrock" as a provider API type.
CHANGELOG.md Add unreleased changelog entry for the Bedrock provider.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +176 to +191
(defn ^:private read-fully
"Reads exactly `n` bytes from `is`. Returns the byte-array, nil on a clean
EOF before any byte was read, or throws on a truncated frame."
^bytes [^InputStream is ^long n]
(let [buf (byte-array n)
first-read (.read is buf 0 n)]
(cond
(neg? first-read) nil
:else (loop [off (long first-read)]
(if (< off n)
(let [r (.read is buf off (- n off))]
(if (neg? r)
(throw (ex-info "Unexpected EOF in Bedrock event-stream frame" {}))
(recur (+ off (long r)))))
buf)))))

Comment on lines +155 to +164
(shared/deep-merge
(assoc-some
{:messages messages
:inferenceConfig {:maxTokens (or max-output-tokens default-max-output-tokens)}}
:system (when-not (string/blank? instructions) [{:text instructions}])
:toolConfig (->tool-config tools)
:additionalModelRequestFields (when reason?
{:reasoning_config {:type "enabled"
:budget_tokens default-reasoning-budget-tokens}}))
(select-keys extra-payload allowed-extra-payload-keys)))
@ericdallo
Copy link
Copy Markdown
Member

@ricardosllm At first glance looks good to me, LMK if you wanna check copilot comments or we are good to go, after release, we might wanna mention in #eca channel in Clojurians to let more people aware of this new integration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New LLM provider AWS bedrock - support converse and conversestream

3 participants