[Bug]: Bedrock Guardrail modified outputs for blocked responses are sent to the model with exceptions disabled

[willdlange]

What happened?

"By default, when Bedrock guardrails block content, LiteLLM raises an HTTP 400 exception. However, you can disable this behavior by setting disable_exception_on_block: true" - https://docs.litellm.ai/docs/proxy/guardrails/bedrock#disabling-exceptions-on-bedrock-block

Using bedrock guardrails in litellm causes issues in client applications such as claude-code and open web ui when disable_exception_on_block is set to true. When this setting is set to false however, litellm will unnecessarily invoke the model with the intercepted message for blocked prompts from the Guardrail, leading to unnecessary token usage.

STEPS TO REPRODUCE

1. Configure Bedrock Guardrail with a message shown for blocked prompts, for example "I can not answer this question"
2. Add guardrail to LiteLLM
   mode: [pre_call, post_call]
   disable_exception_on_block: true
   default_on: true
3. Invoke model with prompt that should be blocked
   curl https://${host}/v1/chat/completions -H "Authorization: Bearer ${api_key}" -H "Content-Type: application/json" -d '{"model": "bedrock-claude-3.7-sonnet","messages": [{"role": "user", "content": "${Restricted prompt}"}],"guardrails": ["${guardrail_name}"]}'

Observed behavior
Prompt -> guardrail intercepted -> guardrail response -> model -> model response -> user

The guardrail successfully intercepts the blocked prompt, but instead of responding with the configured message, the configured message is sent to the llm

Expected behavior
Prompt -> guardrail intercepted -> guardrail response -> user

"When exceptions are disabled, instead of receiving an error, you'll get a successful response containing the Bedrock guardrail's modified/blocked output." -> https://docs.litellm.ai/docs/proxy/guardrails/bedrock#disabling-exceptions-on-bedrock-block

Relevant log output

"guardrail_information": {
    "duration": 0.788103,
    "end_time": 1753873582.723875,
    "start_time": 1753873581.935773,
    "guardrail_mode": [
      "pre_call",
      "post_call"
    ],
    "guardrail_name": "br-guardrail",
    "guardrail_status": "success",
    "guardrail_response": {
      "model": "bedrock-claude-3.7-sonnet",
      "messages": [
        {
          "role": "user",
          "content": "I can not answer this question"
        }
      ]

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.74.7

Twitter / LinkedIn details

No response

[ishaan-jaff]

@willdlange why set disable_exception_on_block for the pre-call ? If you remove it then litellm will just raise an exception with the bedrock guard response

[willdlange]

@willdlange why set disable_exception_on_block for the pre-call ? If you remove it then litellm will just raise an exception with the bedrock guard response

Apologies, I have incorrectly mentioned in the original summary that setting 'disable_exception_on_block = true' causes issues in applications such as claude-code and open web ui. Rather, its causing issues when left as default (disable_exception_on_block = false).

This is the reason why we set 'disable_exception_on_block = true'.

Behavior in each scenario if a guardrail blocks a response:
With disable_exception_on_block = false (default)
Prompt -> guardrail intercepted -> guardrail response (HTTP 400 exception)

Downside here is the exception itself, which disrupts the user experience in Openwebui & claude-code

With disable_exception_on_block = true
Prompt -> guardrail intercepted -> guardrail response -> model -> model response -> user

No exception, but the guardrail response is being sent to the model instead of just being returned to the user.

The desired or expected behavior would be:
Prompt -> guardrail intercepted -> guardrail response (no exception)

[lukasz-wolanski]

@ishaan-jaff my understanding of the flag disable_exception_on_block would be that it only changes if the observable effect for the calling party is either exception or message from the guardrail. A pre-call, blocking Guardrail should never be passed further to llm.

Able to repro

[estohlmann]

@krrishdholakia I see you pushed some code that looks promising for fixing this, any thoughts on when this will be released?

[varadhvi]

@krrishdholakia Thank you, can you confirm whether this is released into the major version?

Fixed on main - b90e916

This should be live in the latest pre-release - https://docs.litellm.ai/release_notes/v1-79-0

[willdlange]

Thank you @krrishdholakia
We have started testing the fix. Using Main latest - https://github.com/berriai/litellm/pkgs/container/litellm/555840307?tag=main-latest

Our test guardrail config:

guardrails:
  - guardrail_name: "bedrock-guardrail"
    litellm_params:
      guardrail: bedrock
      mode: pre_call
      guardrailIdentifier: os.environ/GUARDRAIL_ID
      aws_region_name: eu-west-1
      guardrailVersion: "DRAFT"
      disable_exception_on_block: true
      default_on: true 

Currently any call that triggers the guardrail's blocked action, with disable_exception_on_block=true, returns the following exception
"litellm.exceptions.GuardrailInterventionNormalStringError: Sorry, I cannot answer this question."

• litellm/litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py

  Lines 504 to 507 in b90e916

  	if self.disable_exception_on_block is True:
  	return GuardrailInterventionNormalStringError(
  	message=bedrock_guardrail_output_text
  	)