[Bug]: Router retries "BadRequestError" and other 4xx exceptions despite them being non-retryable

[iotanum]

Summary

The router's retry mechanism in async_function_with_retries() retries all exceptions (including BadRequestError, AuthenticationError, etc.) as long as healthy
deployments exist, ignoring standard HTTP semantics where 4xx client errors should not be retried.

Current Behavior

When raising litellm.BadRequestError (400) from a custom LLM provider, it gets retried num_retries times despite 400 being a client error that should never be retried.

The should_retry_this_error() method at router.py:4801-4865 only explicitly checks for:

• NotFoundError (always raised)
• ContextWindowExceededError (conditional)
• ContentPolicyViolationError (conditional)
• RateLimitError (conditional)
• AuthenticationError (conditional)

BadRequestError and other 4xx errors are not checked, so they pass through to the retry loop which blindly retries all exceptions.

Steps to Reproduce

# In a custom LLM provider
from litellm import BadRequestError

class MyCustomLLM(BaseLLM):
    def completion(self, messages, **kwargs):
        if some_validation_fails:
            raise BadRequestError(
                message="Invalid request",
                model="my-model",
                llm_provider="my-provider"
            )

With proxy config:

router_settings:
  num_retries: 3

Expected: BadRequestError is raised immediately without retries
Actual: BadRequestError is retried 3 times before being raised

Root Cause

In router.py:4862-4863:

if _num_healthy_deployments <= 0:
    raise error
return True  # Allows retry for ALL other exceptions

This means any exception not explicitly listed gets retried if healthy deployments exist.

Suggested Fix

Add proper status code checking in should_retry_this_error() to align with standard HTTP retry semantics:

def should_retry_this_error(self, error, ...):
    # ... existing checks ...

    # Check status code for standard HTTP semantics
    status_code = getattr(error, "status_code", None)
    if status_code is not None:
        # 4xx client errors (except 408, 429) should not be retried
        if 400 <= status_code < 500 and status_code not in (408, 429):
            raise error

    # Also check by exception type
    if isinstance(error, (
        litellm.BadRequestError,      # 400
        litellm.AuthenticationError,  # 401
        litellm.PermissionDeniedError, # 403
        litellm.UnprocessableEntityError, # 422
    )):
        raise error

    # ... rest of existing logic ...

This matches the existing _should_retry() logic in utils.py:6198-6224 which correctly identifies retryable status codes (408, 409, 429, 500+).

Workaround

Currently the only workarounds are:

1. Raise NotFoundError for everything (semantically incorrect)

Not ideal for production use where you want standard HTTP error semantics.

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

1.80.15.rc1

Twitter / LinkedIn details

No response