Feature Request: Option to Export Annotated Score Values and Comments from Human Annotation Queues

[Neha-Jacob]

Describe the feature or potential improvement

Currently, after completing a Human Annotation Queue in Langfuse, users can export the items and their associated metadata such as trace IDs and inputs. However, there is no option to export the actual annotated results such as the annotated score values, comments, and other reviewer-provided fields.

Problem / Limitation

The main purpose of human annotation is to generate high-quality labeled data. While Langfuse provides excellent tools for managing annotation queues and workflows, the lack of an export option for annotated results limits users from:

• Reusing annotated data for benchmarking or evaluation purposes.
• Integrating annotated results into external pipelines, datasets, or evaluation frameworks.
• Conducting offline analysis or custom reporting on human feedback and annotation consistency.

Proposed Solution

Add an Export Annotated Results option in the UI or API that allows users to download:

• Annotated score values
• Annotation comments
• Associated metadata such as trace IDs, annotator info, and timestamps

The export could be in CSV or JSON format, including both the original queue items and their final annotation outcomes.

Use Case

For example, after completing a human annotation queue for chat evaluations, I want to export the final annotated dataset including scores and comments to build a benchmark dataset for future model comparison or automated evaluation pipelines.

Value

• Enables benchmark dataset creation
• Improves data transparency and reproducibility
• Supports custom analytics and reporting workflows
• Makes Langfuse annotation data more portable and useful downstream

Additional information

#9366 is also a similar feature request but for annotation comments only though

[marliessophie]

Agree this could be helpful! Thanks for sharing feedback. Not roadmap commitment I can make at the minute but will update here should we start to develop in this direction.