Allow filtering by environment in score analytics

[marcjaner]

Describe the feature or potential improvement

I'm aware that score analytics is still in beta phase. I've been experimenting with it a bit and found out that I had the need to be able to filter which scores are used for the analytics. More specifically, filter by the environment of the object.

Additional information

No response

[FroeMic]

Hi @marcjaner,

thank your for the feedback!

Could you help me better understand your use-case? I would be interested to understand how you are currently tracking different environments within langfuse?

Best
Michael

[marcjaner]

Of course! We use environments to identify where each trace is generated from. Our current environments are:

• local
• staging
• production

This setup allows us to test changes to our agents in local or staging before deploying them to production. However, these test traces often don’t represent real user behavior. When they’re included in score analytics, they can distort the metrics and give a biased view of model performance.

For example, we have a customer support agent with a session evaluator that classifies the contact reason. This evaluator runs in our backend, and we submit its scores through the scoring API. When we debug or reproduce an issue, we generate multiple traces in the local and staging environments. If these are mixed with production data, they skew the analytics, especially if we’re testing a specific contact reason repeatedly.

Being able to filter by environment and/or tags in score analytics would allow us to exclude these test traces and get a more accurate view of production performance.

[FroeMic]

Thanks for clarifying! We will look into it for the next overhaul!

With our current data structure running this query will not be efficiently possible, so we will need some work on that before we can ship this.

[marcjaner]

Sounds good. I would consider it a nice-to-have, so no hurry there! Thanks 😄