This repository is for data and code accompanying the paper:
ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations
Valentina Pyatkin, Jena D. Hwang, Vivek Srikumar, Ximing Lu, Liwei Jiang, Yejin Choi, Chandra Bhagavatula
ACL 2023
In delta-Clarify we provide the crowdsourced clarification questions.
0. **id** Enumeration of the instances.
1. **source** Whether the questions have been crowdsourced or come from a LLM.
2. **situation** The social or moral situation.
3. **question** The clarification question.
In delta-Clarify-silver we provide the davinci-002 generated questions, given the defeasible SocialChemistry data.
0. **DataSource** Source of the data.
1. **Hypothesis** The social or moral situation together with a judgment.
2. **Update** A weakening or strengthening update.
3. **UpdateType** Whether the update weakens or strengthens the hypothesis.
4. **question_davinci** The question generated by GPT3.
5. **situation** The social or moral situation without the judgment (automatically removed).
We provide a demo of our clarification question generation system.
- Resources on this page are licensed through Apache 2.0.
- Please cite the following paper if you use the data:
@inproceedings{pyatkin2023clarifydelphi,
title={clarifydelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations},
author={Pyatkin, Valentina and
Hwang, Jena D. and
Srikumar, Vivek and
Lu, Ximing and
Jiang, Liwei and
Choi, Yejin and
Bhagavatula, Chandra
},
booktitle={Proceedings of the Association for Computational Linguistics: ACL 2023},
address = "Toronto",
publisher = "Association for Computational Linguistics",
year={2023}
}
If you use the data, please also be sure to cite all of the original datasets on which we built our dataset.
Forbes et al., 2020. Social Chemistry 101: Learning to Reason about Social and Moral Norms
Rudinger et al., 2020. Thinking Like a Skeptic: Defeasible Inference in Natural Language