Skip to content

Consider offloading caching to LiteLLM #492

@eb8680

Description

@eb8680

Currently handlers.llm contains a somewhat half-baked implementation of API call caching in handlers.llm.providers. We should offload all of this to LiteLLM's built-in caching mechanism rather than trying to maintain it ourselves unless it becomes clear that we need some additional behavior.

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions