Feature Request: Show Diff Between Expected and Actual Output in Dataset Run

[yixiaoshen-beep]

Describe the feature or potential improvement

Current limitation

• During a dataset run, there is currently no way to view the differences between the expected and actual outputs.

Requested feature

• Add the ability to highlight differences between expected and actual outputs for dataset runs.
• Support visual diffing, similar to git diff or via other algorithms, to make mismatches easier to identify.

This feature would improve the evaluation experience and help users quickly spot and understand discrepancies. Thank you!

Additional information

No response

[marliessophie]

Hey, thanks for reporting this! Adding support is on the roadmap for the next few weeks, I will keep you posted here when we add this.