Large Language Models are Capable of Offering Cognitive Reappraisal, if Guided

We publicly release our code, model outputs, and expert psychologists’ evaluation data. If you find our work useful or relevant to your research, please consider citing our paper.

Title: Large Language Models are Capable of Offering Cognitive Reappraisal, if Guided

Authors: Hongli Zhan, Allen Zheng, Yoon Kyung Lee, Jina Suh, Junyi Jessy Li, Desmond C. Ong

@misc{zhan2024large,
      title={Large Language Models are Capable of Offering Cognitive Reappraisal, if Guided}, 
      author={Hongli Zhan and Allen Zheng and Yoon Kyung Lee and Jina Suh and Junyi Jessy Li and Desmond C. Ong},
      year={2024},
      eprint={2404.01288},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

TL;DR

Using a series of psychologically grounded constitutions entitled RESORT, we conduct a first-of-its-kind expert psychologist evaluation to explore LLMs’ zero-shot ability to generate cognitive reappraisal responses.

Abstract

Large language models (LLMs) have offered new opportunities for emotional support, and recent work has shown that they can produce empathic responses to people in distress. However, long-term mental well-being requires emotional self-regulation, where a one-time empathic response falls short. This work takes a first step by engaging with cognitive reappraisals, a strategy from psychology practitioners that uses language to targetedly change negative appraisals that an individual makes of the situation; such appraisals is known to sit at the root of human emotional experience. We hypothesize that psychologically grounded principles could enable such advanced psychology capabilities in LLMs, and design RESORT which consists of a series of reappraisal constitutions across multiple dimensions that can be used as LLM instructions. We conduct a first-of-its-kind expert evaluation (by clinical psychologists with M.S. or Ph.D. degrees) of an LLM’s zero-shot ability to generate cognitive reappraisal responses to medium-length social media messages asking for support. This fine-grained evaluation showed that even LLMs at the 7B scale guided by RESORT are capable of generating empathic responses that can help users reappraise their situations.

Repository Status

This repository is under construction! We will frequently maintain and update it. Please let us know if you have any questions by emailing us (honglizhan@utexas.edu) or creating issues in the repo.