[Bug]: Configuring Pass-Through Endpoints through the UI interface leads to memory leaks.

[harvardfly]

What happened?

Configuring Pass-Through Endpoints through the UI interface causes continuous growth in service memory, leading to memory leak issues. This problem has been identified in versions from v1.51.0-stable to v1.68.0-stable. We hope this issue can be resolved as soon as possible.

Relevant log output

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.51.0-v1.68.0

Twitter / LinkedIn details

No response

[harvardfly]

@ishaan-jaff @krrishdholakia Please pay attention to this issue，Configuring Pass-Through Endpoints through the UI，Memory will continue to grow.

[andy-cloudticity]

Experiencing the same issue here with v1.73.0.

[ishaan-jaff]

Ack @andy-cloudticity did you set this up on LiteLLM UI ?

[andy-cloudticity]

This is running on AWS Fargate. I have a single pass-through proxy configuration for Anthropic (that's not being used at the moment).
If I delete this one configuration the memory levels out.