LLM-as-a-Judge on observation

[Lunik]

Describe the feature or potential improvement

My observation span structure can vary a lot from a trace to another. Filtering only by trace attributs is not useful in this case.

Could it be possible Evaluators with "Observations" as Target ? This way I will be able to filter on "type=tools" and "name=XXX"

Additional information

No response

[marliessophie]

Hey, we have a feature request tracking observation-level evals, we will update here when we start working on this/release it in beta. Hope this helps!