Are Langfuse features supported for Livekit realtime agents?

[howard-bailey-nova]

Describe your question

Hi Langfuse team,

First of all, thank you so much for your wonderful product. We currently have a project setup with the livekit integration and are exploring how we can make the best use of Langfuse's features.

The key features we are looking at, are as follows:

1. Agent observability metrics via dashboards.
2. Playground
3. LLM-as-judge

We are currently having some issues integrating with these features, and I wanted to post here to understand whether these features are compatible with the Livekit integration (when using realtime models) and if there is a specific recommended setup.

I have reviewed the following documentation so far:

1. Langfuse docs on integrating with Livekit - https://langfuse.com/integrations/frameworks/livekit
2. Langfuse example trace of an integrated Livekit product - Link
3. Livekit documentation on observability data it exposes - https://docs.livekit.io/deploy/observability/data/

These are the challenges so far:

1. Our latency values seem incorrect on traces in our project - all spans seem to have latency, even those that represent frontend operations like user_turn (same is true of the Langfuse example linked above)
2. Our latency values seem incorrect in Langfuse managed dashboards in our project - traces that represent actions on the frontend are being counted towards latency like user_turn
3. Traces don't seem to represent agent generation time, but instead seem to represent speaking time
4. User input and agent output are displayed on separate traces, making it hard to export to playground or use with LLM-as-jusge.

So my questions are:

1. Are the Dashboard, Playground and LLM-as-judge features compatible with Livekit agent models
2. If yes, is there a recommended approach to ensuring traces are compatible?
3. Are there any other resources or aspects we should be aware of?

Many thanks for your time in reading this query, and for any help you are able to provide.

Note: I posted this query first in Langfuse AI and it asked me to raise to contact support.

Langfuse Cloud or Self-Hosted?

Langfuse Cloud

If Self-Hosted

N/A

If Langfuse Cloud

https://langfuse.novakidschool.com/project/cmi7ta20002k3ob0792uuboxt/traces

SDK and integration versions

No response

Pre-Submission Checklist

• I have checked for existing issues/discussions and consulted Langfuse AI.

[jannikmaierhoefer]

Hi @howard-bailey-nova, thanks for the feedback!

> Our latency values seem incorrect on traces in our project - all spans seem to have latency, even those that represent frontend operations like user_turn (same is true of the Langfuse example linked above)

These latency values are directly reported by the livekit otel exporter. Please open an issue in livekits repository if you think these values are incorrect.

> Our latency values seem incorrect in Langfuse managed dashboards in our project - traces that represent actions on the frontend are being counted towards latency like user_turn

You might be able to create a custom dashboard that only shows the average latencies of the agent part (i.e. only filtering for observations with the nane agent_turn). Could you check if this works?

> Traces don't seem to represent agent generation time, but instead seem to represent speaking time

Could you check if showing the debug spans that are currently hidden reveals this information (screenshot below)?

> User input and agent output are displayed on separate traces, making it hard to export to playground or use with LLM-as-jusge.

Did you mean that they are on different observations? If tracing is set up correctly, user input and agent output should already be on the same trace.

We are also currently working on enabling more support for multimodal and voice applications across the product (Playground, Experiments, etc. )