[Bug]: LiteLLM Bedrock Guardrail Error Response Incomplete

[duodecanol]

What happened?

1. Summary of Bug:

The LiteLLM integration with Bedrock guardrails is returning an incomplete response body when a guardrail policy is violated. The actual response's error.message.bedrock_guardrail_response field contains only a string message (e.g., "Sorry, the model cannot answer this question.") instead of the expected full JSON object detailed in the documentation, which should include critical fields like assessments, output, outputs, and usage.

---

2. Steps to Reproduce:

1. Configure LiteLLM to use a Bedrock model with a pre-configured guardrail.
2. Ensure the guardrail is set up to block a specific input (e.g., a "coffee" topic policy as per the provided documentation example).
3. Make an API call to LiteLLM with a prompt that is expected to trigger the configured Bedrock guardrail (e.g., a prompt containing the word "coffee").
4. Observe the returned API response.

---

3. Expected Behavior:

Based on the LiteLLM documentation, when a Bedrock guardrail policy is violated, the API response should contain the full bedrock_guardrail_response object within error.message, including detailed information such as:

{
  "error": {
    "message": {
      "error": "Violated guardrail policy",
      "bedrock_guardrail_response": {
        "action": "GUARDRAIL_INTERVENED",
        "assessments": [
          {
            "topicPolicy": {
              "topics": [
                {
                  "action": "BLOCKED",
                  "name": "Coffee",
                  "type": "DENY"
                }
              ]
            }
          }
        ],
        "blockedResponse": "Sorry, the model cannot answer this question. coffee guardrail applied ",
        "output": [
          {
            "text": "Sorry, the model cannot answer this question. coffee guardrail applied "
          }
        ],
        "outputs": [
          {
            "text": "Sorry, the model cannot answer this question. coffee guardrail applied "
          }
        ],
        "usage": {
          "contentPolicyUnits": 0,
          "contextualGroundingPolicyUnits": 0,
          "sensitiveInformationPolicyFreeUnits": 0,
          "sensitiveInformationPolicyUnits": 0,
          "topicPolicyUnits": 1,
          "wordPolicyUnits": 0
        }
      }
    },
    "type": "None",
    "param": "None",
    "code": "400"
  }
}

Specifically, the assessments field is crucial for understanding which policy was violated, and usage provides valuable insights into the guardrail's operation.

---

4. Actual Behavior:

When a Bedrock guardrail policy is violated, the bedrock_guardrail_response field within error.message is an unparsed string instead of a JSON object. This results in a loss of critical information provided by the Bedrock guardrail, such as assessments and usage data.

Actual Response Received:

{
  "error": {
    "message": "{'error': 'Violated guardrail policy', 'bedrock_guardrail_response': 'Sorry, the model cannot answer this question.'}",
    "type": "None",
    "param": "None",
    "code": "400"
  }
}

---

5. Impact:

• Degraded Error Handling: Developers cannot programmatically determine the specific reason for a guardrail intervention (e.g., which policy was triggered, which topic was blocked) without parsing an embedded string, which is fragile and error-prone.
• Missing Observability: The absence of usage data prevents proper monitoring and cost analysis related to guardrail interactions.
• Inconsistent API Contract: The actual response deviates significantly from the documented structure, making integration and error handling more complex and less reliable.

---

6. Suggested Fix (Optional):

The bedrock_guardrail_response field should be correctly parsed into a JSON object before being returned in the LiteLLM error message. It appears the inner JSON object is being stringified within the outer JSON message.

---

7. Environment:

• Operating System: Ubuntu 24.10
• LiteLLM Version: v1.74.7-nightly
• LiteLLM Guardrail mode: pre_call, post_call, during_call
• Guardrail Configuration:
  • Cross-Region inference [US Guardrail v1:0]

---

Relevant log output

litellm-1  | 03:19:12 - LiteLLM Proxy:INFO: parallel_request_limiter.py:68 - Current Usage of user in this minute: None
litellm-1  | 03:19:12 - LiteLLM Router:INFO: router.py:6539 - get_available_deployment for model: gemini/gemini-2.0-flash, Selected deployment: {'model_name': 'gemini/*', 'litellm_params': {'api_key': 'AI**********', 'use_in_pass_through': False, 'use_litellm_proxy': False, 'merge_reasoning_content_in_choices': False, 'model': 'gemini/gemini-2.0-flash', 'litellm_provider': 'gemini', 'drop_params': True}, 'model_info': {'id': 'a03abdd3dbf8532f7b8a43ca5e5f671b141223751697d433967ce00ce8c158e5', 'db_model': False}} for model: gemini/gemini-2.0-flash
litellm-1  | 03:19:12 - LiteLLM:INFO: utils.py:3230 - 
litellm-1  | LiteLLM completion() model= gemini-2.0-flash; provider = gemini
litellm-1  | 03:19:13 - LiteLLM Router:INFO: router.py:1184 - litellm.acompletion(model=gemini/gemini-2.0-flash) 200 OK
litellm-1  | 03:19:13 - LiteLLM Proxy:INFO: db_spend_update_writer.py:157 - disable_spend_logs=True. Skipping writing spend logs to db. Other spend updates - Key/User/Team table will still occur.
litellm-1  | 03:19:13 - LiteLLM:INFO: langfuse.py:286 - Langfuse Layer Logging - logging success
litellm-1  | 03:19:13 - LiteLLM Proxy:INFO: db_spend_update_writer.py:1089 - Logged request status: success
litellm-1  | 03:19:13 - LiteLLM Proxy:INFO: db_spend_update_writer.py:1089 - Logged request status: success
litellm-1  | 03:19:13 - LiteLLM Proxy:INFO: db_spend_update_writer.py:1089 - Logged request status: success
litellm-1  | 03:19:14 - LiteLLM Proxy:ERROR: common_request_processing.py:639 - litellm.proxy.proxy_server._handle_llm_api_exception(): Exception occured - 400: {'error': 'Violated guardrail policy', 'bedrock_guardrail_response': 'Sorry, the model cannot answer this question.'}
litellm-1  | Traceback (most recent call last):
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 3794, in chat_completion
litellm-1  |     return await base_llm_response_processor.base_process_llm_request(
litellm-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1  |     ...<16 lines>...
litellm-1  |     )
litellm-1  |     ^
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/proxy/common_request_processing.py", line 433, in base_process_llm_request
litellm-1  |     responses = await llm_responses
litellm-1  |                 ^^^^^^^^^^^^^^^^^^^
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/proxy/utils.py", line 654, in during_call_hook
litellm-1  |     raise e
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/proxy/utils.py", line 648, in during_call_hook
litellm-1  |     await callback.async_moderation_hook(
litellm-1  |     ...<3 lines>...
litellm-1  |     )
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/integrations/custom_guardrail.py", line 429, in async_wrapper
litellm-1  |     return self._process_error(
litellm-1  |            ~~~~~~~~~~~~~~~~~~~^
litellm-1  |         e=e,
litellm-1  |         ^^^^
litellm-1  |     ...<3 lines>...
litellm-1  |         duration=(datetime.now() - start_time).total_seconds(),
litellm-1  |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1  |     )
litellm-1  |     ^
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/integrations/custom_guardrail.py", line 373, in _process_error
litellm-1  |     raise e
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/integrations/custom_guardrail.py", line 420, in async_wrapper
litellm-1  |     response = await func(*args, **kwargs)
litellm-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py", line 481, in async_moderation_hook
litellm-1  |     bedrock_guardrail_response = await self.make_bedrock_api_request(
litellm-1  |                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1  |         source="INPUT", messages=new_messages, request_data=data
litellm-1  |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1  |     )
litellm-1  |     ^
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py", line 272, in make_bedrock_api_request
litellm-1  |     raise self._get_http_exception_for_blocked_guardrail(
litellm-1  |         bedrock_guardrail_response
litellm-1  |     )
litellm-1  | fastapi.exceptions.HTTPException: 400: {'error': 'Violated guardrail policy', 'bedrock_guardrail_response': 'Sorry, the model cannot answer this question.'}
litellm-1  | 03:19:14 - LiteLLM:ERROR: langfuse.py:290 - Langfuse Layer Error(): Exception occured - HTTPException.__init__() missing 1 required positional argument: 'status_code'
litellm-1  | Traceback (most recent call last):
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 3794, in chat_completion
litellm-1  |     return await base_llm_response_processor.base_process_llm_request(
litellm-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1  |     ...<16 lines>...
litellm-1  |     )
litellm-1  |     ^
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/proxy/common_request_processing.py", line 433, in base_process_llm_request
litellm-1  |     responses = await llm_responses
litellm-1  |                 ^^^^^^^^^^^^^^^^^^^
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/proxy/utils.py", line 654, in during_call_hook
litellm-1  |     raise e
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/proxy/utils.py", line 648, in during_call_hook
litellm-1  |     await callback.async_moderation_hook(
litellm-1  |     ...<3 lines>...
litellm-1  |     )
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/integrations/custom_guardrail.py", line 429, in async_wrapper
litellm-1  |     return self._process_error(
litellm-1  |            ~~~~~~~~~~~~~~~~~~~^
litellm-1  |         e=e,
litellm-1  |         ^^^^
litellm-1  |     ...<3 lines>...
litellm-1  |         duration=(datetime.now() - start_time).total_seconds(),
litellm-1  |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1  |     )
litellm-1  |     ^
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/integrations/custom_guardrail.py", line 373, in _process_error
litellm-1  |     raise e
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/integrations/custom_guardrail.py", line 420, in async_wrapper
litellm-1  |     response = await func(*args, **kwargs)
litellm-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py", line 481, in async_moderation_hook
litellm-1  |     bedrock_guardrail_response = await self.make_bedrock_api_request(
litellm-1  |                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1  |         source="INPUT", messages=new_messages, request_data=data
litellm-1  |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1  |     )
litellm-1  |     ^
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py", line 272, in make_bedrock_api_request
litellm-1  |     raise self._get_http_exception_for_blocked_guardrail(
litellm-1  |         bedrock_guardrail_response
litellm-1  |     )
litellm-1  | fastapi.exceptions.HTTPException: 400: {'error': 'Violated guardrail policy', 'bedrock_guardrail_response': 'Sorry, the model cannot answer this question.'}
litellm-1  | 
litellm-1  | During handling of the above exception, another exception occurred:
litellm-1  | 
litellm-1  | Traceback (most recent call last):
litellm-1  |   File "/usr/lib/python3.13/site-packages/litellm/integrations/langfuse/langfuse.py", line 224, in log_event_on_langfuse
litellm-1  |     optional_params = copy.deepcopy(kwargs.get("optional_params", {}))
litellm-1  |   File "/usr/lib/python3.13/copy.py", line 137, in deepcopy
litellm-1  |     y = copier(x, memo)
litellm-1  |   File "/usr/lib/python3.13/copy.py", line 222, in _deepcopy_dict
litellm-1  |     y[deepcopy(key, memo)] = deepcopy(value, memo)
litellm-1  |                              ~~~~~~~~^^^^^^^^^^^^^
litellm-1  |   File "/usr/lib/python3.13/copy.py", line 163, in deepcopy
litellm-1  |     y = _reconstruct(x, memo, *rv)
litellm-1  |   File "/usr/lib/python3.13/copy.py", line 260, in _reconstruct
litellm-1  |     state = deepcopy(state, memo)
litellm-1  |   File "/usr/lib/python3.13/copy.py", line 137, in deepcopy
litellm-1  |     y = copier(x, memo)
litellm-1  |   File "/usr/lib/python3.13/copy.py", line 222, in _deepcopy_dict
litellm-1  |     y[deepcopy(key, memo)] = deepcopy(value, memo)
litellm-1  |                              ~~~~~~~~^^^^^^^^^^^^^
litellm-1  |   File "/usr/lib/python3.13/copy.py", line 137, in deepcopy
litellm-1  |     y = copier(x, memo)
litellm-1  |   File "/usr/lib/python3.13/copy.py", line 222, in _deepcopy_dict
litellm-1  |     y[deepcopy(key, memo)] = deepcopy(value, memo)
litellm-1  |                              ~~~~~~~~^^^^^^^^^^^^^
litellm-1  |   File "/usr/lib/python3.13/copy.py", line 163, in deepcopy
litellm-1  |     y = _reconstruct(x, memo, *rv)
litellm-1  |   File "/usr/lib/python3.13/copy.py", line 254, in _reconstruct
litellm-1  |     y = func(*args)
litellm-1  | TypeError: HTTPException.__init__() missing 1 required positional argument: 'status_code'
litellm-1  | 03:19:14 - LiteLLM Proxy:INFO: db_spend_update_writer.py:157 - disable_spend_logs=True. Skipping writing spend logs to db. Other spend updates - Key/User/Team table will still occur.
litellm-1  | 03:19:14 - LiteLLM Proxy:INFO: db_spend_update_writer.py:1089 - Logged request status: failure
litellm-1  | 03:19:14 - LiteLLM Proxy:INFO: db_spend_update_writer.py:1089 - Logged request status: failure
litellm-1  | 03:19:14 - LiteLLM Proxy:INFO: db_spend_update_writer.py:1089 - Logged request status: failure
litellm-1  | 172.19.0.1:51532 - "POST /v1/chat/completions HTTP/1.1" 400
litellm-1  | 03:19:15 - LiteLLM Proxy:INFO: db_spend_update_writer.py:961 - Processed 2 daily user transactions in 0.01s
litellm-1  | 03:19:15 - LiteLLM Proxy:INFO: db_spend_update_writer.py:961 - Processed 2 daily team transactions in 0.01s
litellm-1  | 03:19:15 - LiteLLM Proxy:INFO: db_spend_update_writer.py:961 - Processed 2 daily tag transactions in 0.01s

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.74.7-nightly

Twitter / LinkedIn details

No response

[0x-fang]

I had reproduced the issue, but it can be resolved by enabling the trace in the API call (e.g., add "trace": "enabled" in guardrailConfig)

First, create a guardrail in bedrock to deny topic of Coffee as the example in documents. Here is the sample code:

response = completion(
    model="us.anthropic.claude-3-7-sonnet-20250219-v1:0",
    messages=[
        {
            "content": "where do i buy Coffee from? ",
            "role": "user",
        }
    ],
    max_tokens=10,
    guardrailConfig={
        "guardrailIdentifier": <Id>, # The identifier (ID) for the guardrail.
        "guardrailVersion": "DRAFT",           # The version of the guardrail.
        "trace": "enabled",                   # The trace behavior for the guardrail. Can either be "disabled" or "enabled"
    },
)

print(response)

It generates the output like below, which contains the inputAssessment and usage fields that users are most interested.

ModelResponse(
  id='chatcmpl-364940be-50bd-4009-834b-b3704ee6ccc5', 
  created=1755726524, 
  model='us.anthropic.claude-3-7-sonnet-20250219-v1:0', 
  object='chat.completion', 
  system_fingerprint=None, 
  choices=[
    Choices(finish_reason='guardrail_intervened', 
                  index=0, 
                  message=Message(
                    content='Sorry, the model cannot answer this question.', 
                    role='assistant', tool_calls=None, function_call=None, provider_specific_fields=None))
  ], 
  usage=Usage(completion_tokens=0, prompt_tokens=0, total_tokens=0, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None), cache_creation_input_tokens=0, cache_read_input_tokens=0), 
  trace={'guardrail': 
    {'actionReason': 'Guardrail blocked.', 
     'inputAssessment': {
       'epnoq50yitim': {
         'invocationMetrics': {
            'guardrailCoverage': {'textCharacters': {'guarded': 27, 'total': 28}}, 
            'guardrailProcessingLatency': 277, 
            'usage': {
                  'automatedReasoningPolicies': 0, 
                  'automatedReasoningPolicyUnits': 0, 
                  'contentPolicyImageUnits': 0, 
                  'contentPolicyUnits': 0, 
                  'contextualGroundingPolicyUnits': 0, 
                  'sensitiveInformationPolicyFreeUnits': 0, 
                  'sensitiveInformationPolicyUnits': 0, 
                  'topicPolicyUnits': 1, 
                  'wordPolicyUnits': 1}}, 
            'topicPolicy': {
                  'topics': [
                    {'action': 'BLOCKED', 'detected': True, 'name': 'Coffee', 'type': 'DENY'}]}, 
                     'wordPolicy': {'customWords': [{'action': 'BLOCKED', 'detected': True, 'match': 'Coffee'}]}}}, 'modelOutput': []}
     }
)

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.