The project deals with the Quality of Experience of machine-generated texts. The focus lies on the two text types Machine Translation and Automatic Text Summarization. The goals of the project are to identify perceptive quality dimensions, to provide subjective methods for the quantification of the quality dimensions, to determine automatically extractable factors that correlate with text quality, and to develop prediction models that can assess the overall quality of a machine-generated text.
The folder "MT Quality Dimensions" contains datasets for the identification and quantification of quality dimensions for Machine Translation.
The folder "SUM Quality Dimensions" contains datasets for the identification and quantification of quality dimensions for Automatic Text Summarization.
Please cite the following publication:
Vivien Macketanz, Babak Naderi, Steven Schmidt, and Sebastian Möller. 2022. Perceptual Quality Dimensions of Machine-Generated Text with a Focus on Machine Translation. In: Proceedings of the Workshop on Human Evaluation of NLP Systems (HumEval). Association for Computational Linguistics. (to appear)