Welcome to DeltaScore: Fine-Grained Story Evaluation with Perturbations, which has been published in EMNLP2023.
The goal of this project is to propose a new methodology to conduct fine-grained story evaluation by differentiating perturbations. We have demonstrated that DeltaScore can outperform evaluation methods that use likelihood directly, and you can find the details in our published paper.
If you have any questions, feedback, feel free to reach out to Zhuohan Xie at zhuohanx@student.unimelb.edu.au. We welcome any inquiries and look forward to hearing from you!