Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 36 additions & 9 deletions lib/ruby_llm/providers/bedrock.rb
Original file line number Diff line number Diff line change
Expand Up @@ -127,26 +127,51 @@ def model_supports_top_k?(model)
# Returns true if the InvokeModel protocol should be used for this model.
# `bedrock_use_invoke_model` can be:
# - false / nil → always Converse (default)
# - true → InvokeModel for all Anthropic models
# - true → InvokeModel for all verifiably Anthropic models
# - Array → InvokeModel when model.id is in the list
# - Proc/lambda → InvokeModel when the callable returns truthy for model
#
# Vendor verification interacts with the selector in two tiers:
# - Ids that are provably non-Anthropic (a known vendor prefix like amazon./meta.,
# with or without a cross-region geo prefix) are never routed, under any selector —
# the InvokeModel payload is Anthropic Messages format and would be rejected.
# - Ids that cannot be verified either way — chiefly application-inference-profile
# ARNs, whose Model::Info carries no provider_name metadata on the
# assume_model_exists path — are routed when the selector opts in explicitly
# (Array or Proc). An operator naming the exact id IS the verification. Only the
# blanket `true` selector requires positive verification via anthropic_model?,
# because it expresses "all Anthropic models", not "this specific model".
def invoke_model?(model)
selector = @config.bedrock_use_invoke_model
return false unless selector
return false unless anthropic_model?(model)
return false if non_anthropic_model?(model)

case selector
when true
true
anthropic_model?(model)
when Array
selector.include?(model.id)
else
selector.respond_to?(:call) ? selector.call(model) : false
end
end

NON_ANTHROPIC_PREFIXES = %w[amazon. meta. ai21. cohere. mistral. writer. stability.].freeze
private_constant :NON_ANTHROPIC_PREFIXES
NON_ANTHROPIC_VENDORS = %w[amazon meta ai21 cohere mistral writer stability].freeze
private_constant :NON_ANTHROPIC_VENDORS

# Matches known non-Anthropic vendor ids in both bare ("amazon.nova-pro-v1:0") and
# cross-region ("us.amazon.nova-pro-v1:0") forms.
NON_ANTHROPIC_PATTERN = /\A(?:[a-z0-9-]+\.)?(?:#{NON_ANTHROPIC_VENDORS.join('|')})\./

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the optional geo-prefix segment [a-z0-9-]+ in NON_ANTHROPIC_PATTERN is unboundedly wide — it would treat any single-dot-prefixed string (e.g. a hypothetical "foo.amazon.x") as a cross-region non-Anthropic id. The existing cross-region geo codes are all short (us, eu, apac, global, jp, au, us-gov), so a tighter bound like [a-z]{2,6}(?:-[a-z]+)? would make the intent clearer and prevent surprise matches on longer prefixes. Low severity since real Bedrock model IDs don't look like that today.

private_constant :NON_ANTHROPIC_PATTERN

# True only when the id provably belongs to a non-Anthropic vendor. ARNs return false:
# they don't encode the vendor, so they are "unverifiable", not "non-Anthropic".
def non_anthropic_model?(model)
id = model.id.to_s
return false if id.start_with?('arn:')

NON_ANTHROPIC_PATTERN.match?(id)
end

def anthropic_model?(model)
id = model.id.to_s
Expand All @@ -163,13 +188,15 @@ def anthropic_model?(model)
# Application-inference-profile ARNs do not encode the underlying vendor in the ARN
# string itself. When available, consult model.metadata[:provider_name] (populated
# by Bedrock::Models for registered foundation models) to confirm the profile is
# Anthropic-backed. If that field is absent (e.g. a runtime-only ARN the registry
# has never seen), log a warning and refuse to route rather than silently forwarding
# an incompatible payload to a non-Anthropic model.
# Anthropic-backed. If that field is absent (e.g. an assume_model_exists chat, whose
# Model::Info carries no registry metadata), log a warning and refuse to route rather
# than silently forwarding an incompatible payload to a non-Anthropic model — the
# operator can still opt in explicitly via an Array or Proc selector (see
# invoke_model?).
return arn_anthropic_model?(model, id) if id.start_with?('arn:')

# Block known non-Anthropic Bedrock vendor prefixes (Nova, Llama, Jurassic, etc.).
return false if NON_ANTHROPIC_PREFIXES.any? { |pfx| id.start_with?(pfx) }
return false if NON_ANTHROPIC_PATTERN.match?(id)

provider = model.respond_to?(:provider) ? model.provider : nil
provider.to_s == 'anthropic'
Expand Down
38 changes: 38 additions & 0 deletions spec/ruby_llm/providers/bedrock_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,44 @@ def model_double(id, metadata: {}, provider: 'bedrock')
end
end

context 'with an unverifiable ARN id (no provider_name metadata)' do # rubocop:disable RSpec/MultipleMemoizedHelpers
before { allow(RubyLLM.logger).to receive(:warn) }

it 'routes to BedrockInvokeModel when an Array selector lists the ARN' do

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The before { allow(RubyLLM.logger).to receive(:warn) } stub is shared across all four examples in this context, but the three cases that bypass anthropic_model? (Array, Proc-true, Proc-false) never actually trigger the warning path. Consider adding a negative assertion to the first two cases (expect(RubyLLM.logger).not_to have_received(:warn)) to confirm the warning is silenced by bypassing anthropic_model? rather than by coincidence. The test for the true selector case at line 256–260 already does this correctly.

provider = build_bedrock(use_invoke_model: [arn_id])
expect(provider.protocol_for(model_double(arn_id))).to be(RubyLLM::Protocols::BedrockInvokeModel)
end

it 'routes to BedrockInvokeModel when a Proc selector returns true' do
provider = build_bedrock(use_invoke_model: ->(_m) { true })
expect(provider.protocol_for(model_double(arn_id))).to be(RubyLLM::Protocols::BedrockInvokeModel)
end

it 'routes to Converse when a Proc selector returns false' do
provider = build_bedrock(use_invoke_model: ->(_m) { false })
expect(provider.protocol_for(model_double(arn_id))).to be(RubyLLM::Protocols::Converse)
end

it 'still requires positive verification under the blanket true selector' do
provider = build_bedrock(use_invoke_model: true)
expect(provider.protocol_for(model_double(arn_id))).to be(RubyLLM::Protocols::Converse)
expect(RubyLLM.logger).to have_received(:warn).with(/cannot verify.*Anthropic-backed/)
end
end

context 'with provably non-Anthropic ids under explicit selectors' do # rubocop:disable RSpec/MultipleMemoizedHelpers
it 'never routes a bare vendor id, even when an Array selector lists it' do
provider = build_bedrock(use_invoke_model: [nova_id])
expect(provider.protocol_for(model_double(nova_id))).to be(RubyLLM::Protocols::Converse)
end

it 'never routes a cross-region vendor id, even when a Proc selector returns true' do
provider = build_bedrock(use_invoke_model: ->(_m) { true })
expect(provider.protocol_for(model_double(us_nova_id))).to be(RubyLLM::Protocols::Converse)
expect(provider.protocol_for(model_double(llama_id))).to be(RubyLLM::Protocols::Converse)
end
end

describe 'anthropic_model? directly' do # rubocop:disable RSpec/MultipleMemoizedHelpers
let(:bedrock) { build_bedrock }

Expand Down