Integrate LiteLLM into langfuse

[skuchipu]

Hi Team,
I'm using Helmchart to deploy Langfuse , , can someone help on how to integrate existing Litellm into Langfuse ?

Thank you

[dosubot[bot]]

To integrate LiteLLM with Langfuse when deploying via Helm, you’ll need to set up the integration at the application level and configure the required environment variables in your Helm deployment.

There are three supported integration methods: using the LiteLLM Proxy with the Langfuse OpenAI SDK Wrapper (preferred), sending logs from the LiteLLM Proxy to Langfuse, or using the LiteLLM Python SDK to send logs directly to Langfuse. The preferred method captures token counts, latencies, streaming response times, and API errors automatically (docs).

For any of these methods, you must set the LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY environment variables in your deployment. These keys are project-specific and can be found in your Langfuse dashboard under Project → Settings (discussion). In your Helm values file (typically values.yaml), add these keys under the environment section for your Langfuse deployment.

Example:

env:
  LANGFUSE_PUBLIC_KEY: "<your-public-key>"
  LANGFUSE_SECRET_KEY: "<your-secret-key>"

If you’re using the LiteLLM Proxy, you can enable logging to Langfuse in the LiteLLM UI or set up the callback in your LiteLLM configuration. For the Python SDK, ensure you’re using Langfuse Python SDK v2, as v3 is not compatible with LiteLLM (docs).

You’ll need to customize your Helm deployment to include these environment variables and ensure your LiteLLM service is configured to communicate with Langfuse. There’s no official Helm chart for LiteLLM integration, so these changes are user-managed in your deployment process (self-hosting guide).

For more details and code examples, see the Langfuse LiteLLM integration docs and the LiteLLM Proxy logging guide.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[jannikmaierhoefer]

Hi @skuchipu, did Dosu answer your question?

[skuchipu]

Hi , I was looking to add my custom endpoints to LLM connection to be able to use LLM-as-a-judge , my models are deployed using kserve and has integrated through litellm proxy , I 'm looking for an option to either use litellm endpoints or kserve endpoints to add as llm connection , any insights are appreciated .