Skip to content

API_TIMEOUT_MS - Does it also work for litellm? #4968

Description

@skye0402

Environment

  • Platform (select one):
    • Anthropic API
    • AWS Bedrock
    • Google Vertex AI
    • [X ] Other: Litellm pointing to AWS Bedrock
  • Claude CLI version: 1.0.67
  • Operating System: Ubuntu 22.04
  • Terminal: bash

Bug Description

Setup works - but every now end then I get timeouts (usually with non-streaming calls from Claude Code). I see in the litellm proxy logs that the converse API delivers the response but too late, Claude Code already shows the timeout and retries. Of course this is a waste of tokens as I get the response 3-5 times before I'm lucky and it arrives in time. I found the parameter API_TIMEOUT_MS` and set it before calling claude code but it seems to have no effect?

Steps to Reproduce

  1. Set API_TIMEOUT_MS to e.g. 600000.
  2. Run Claude Code
  3. Let it call a an endpoint defined in ANTHROPIC_BASE_URL and measure if the timeout is changed to default.

Expected Behavior

The defined timeout should be honored by Claude Code

Actual Behavior

No change in timeout observed (maybe 30s?)

Additional Context

None

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions