Skip to content

[Bug]: Crossed TPM / RPM / Max Parallel Request Limit #9440

Description

@ma-armenta

What happened?

There are no limits set for any team nor model. However randomly this message starts coming up.

Relevant log output

8:08:37 - LiteLLM Proxy:ERROR: common_request_processing.py:296 - litellm.proxy.proxy_server._handle_llm_api_exception(): Exception occured - 429: LiteLLM Rate Limit Handler for rate limit type = team. Crossed TPM / RPM / Max Parallel Request Limit. current rpm: 1, rpm limit: -2, current tpm: 0, tpm limit: 9223372036854775807, current max_parallel_requests: 1, max_parallel_requests: 9223372036854775807
 Traceback (most recent call last):
   File "/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 3531, in chat_completion
     return await base_llm_response_processor.base_process_llm_request(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     ...<16 lines>...
     )
     ^
   File "/usr/lib/python3.13/site-packages/litellm/proxy/common_request_processing.py", line 164, in base_process_llm_request
     self.data = await proxy_logging_obj.pre_call_hook(  # type: ignore
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         user_api_key_dict=user_api_key_dict, data=self.data, call_type="completion"
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     )
     ^
   File "/usr/lib/python3.13/site-packages/litellm/proxy/utils.py", line 539, in pre_call_hook
     raise e
   File "/usr/lib/python3.13/site-packages/litellm/proxy/utils.py", line 526, in pre_call_hook
     response = await _callback.async_pre_call_hook(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     ...<4 lines>...
     )
     ^
   File "/usr/lib/python3.13/site-packages/litellm/proxy/hooks/parallel_request_limiter.py", line 386, in async_pre_call_hook
     await self.check_key_in_limits(
     ...<11 lines>...
     )
   File "/usr/lib/python3.13/site-packages/litellm/proxy/hooks/parallel_request_limiter.py", line 97, in check_key_in_limits
     raise HTTPException(
     ...<3 lines>...
     )
 fastapi.exceptions.HTTPException: 429: LiteLLM Rate Limit Handler for rate limit type = team. Crossed TPM / RPM / Max Parallel Request Limit. current rpm: 1, rpm limit: -2, current tpm: 0, tpm limit: 9223372036854775807, current max_parallel_requests: 1, max_parallel_requests: 9223372036854775807
 18:08:37 - LiteLLM Proxy:ERROR: _common.py:120 - Giving up chat_completion(...) after 1 tries (litellm.proxy._types.ProxyException)

Are you a ML Ops Team?

Yes

What LiteLLM version are you on ?

v1.63.12-nightly

Twitter / LinkedIn details

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions