Why use helicone?

[krrishdholakia]

SemanticSearch/backend/llm.py

Line 13 in f8189d5

litellm.api_base = "https://oai.hconeai.com/v1"

Hey @nsbradford,

I saw you're logging responses to promptlayer but also using helicone. Curious - why?

If it's for caching - is there something in our implementation you think is missing - https://docs.litellm.ai/docs/caching/

[nsbradford]

AFAIK LiteLLM does not support a managed caching solution, the docs only mention self-managed Redis.

[krrishdholakia]

Tracking this here - BerriAI/litellm#432

Do you want us to provide a hosted caching solution? @nsbradford

[nsbradford]

TBH, not a top priority - writing my own cache also only takes <1 hour and then I don't have to worry about whether the 3rd-party caching middleware is reliable, which is why in practice I tend to write my own cache on larger projects

[nsbradford]

(would be open to it if implemented, though.)