This data set accompanies the research paper, Automating Behavioral Testing in Machine Translation.
It includes the following:
data/prompts_src_gencontains the prompts that were used to generate the English source sentences for behavioral testing, as described in the paper.data/filtered_src_gencontains the generated source sentences after the filter steps outlined in the paper.data/prompts_candidatescontains the prompts that were used to generate the target-language-specific candidate sets.data/candidate_setscontains the generated candidate sets (unfiltered).
If you use this dataset, please cite our paper as follows:
Javier Ferrando, Matthias Sperber, Hendra Setiawan, Dominic Telaar, Saša Hasan (2023). Automating Behavioral Testing in Machine Translation. Conference on Machine Translation (WMT).