The dataset comprises 8,712 files across 6 programming languages, featuring verified tasks and benchmarks for evaluating coding agents and language models. It supports coding agents, language models, and developer tools with verified benchmark scores and multi-language test sets. - Get the data
Characteristic | Data |
---|---|
Description | An extended benchmark of real-world software engineering tasks with enhanced artifacts and broader language coverage |
Data types | Text |
Tasks | Bug fixing, code completion, pull request generation, automated code review |
Total number of files | 8,712 |
Total number of people | 30 |
Labeling | Annotated with golden patches, test patches, post-patch reference states, and metadata stored in parquet files (e.g., repository name, issue/PR identifier, diffs, test results) |
Programming languages | C#, Go, PHP, Rust, Kotlin, Ruby |
π Sample dataset available! For full access, contact us to discuss purchase terms.
- Go - Files in Go
- Scala - Files in Scala