Filter & Sort by non-score columns in Compare View

[rebeccalimxe]

Describe the feature or potential improvement

Currently the Compare view is the only view where human annotation can be done while having the expected output in view. However, it is difficult for multiple people to do human annotation on dataset runs as there is no way to slice or order the dataset items in the run, other than based on scores.

It would be helpful if the compare view can be filtered & sorted by other columns, just like in the trace view, eg. filter by topics or assignees via metadata or sort via id, inputs and outputs. This would allow annotators to collaborate by spliting up the work of annotating datasets and access the right dataset run items if necessary.

An alternative is to have human annotation queues where the expected output can be viewed, if the trace is an experiment run.

Additional information

No response

[marliessophie]

Hey, thanks for sharing! Agree this would be helpful.