Towards Better Chain-of-Thought: A Reflection on Effectiveness and Faithfulness

[Paper]

Description

This repository hosts the codes of our work: "Towards Better Chain-of-Thought: A Reflection on Effectiveness and Faithfulness", which is accepted in the ACL 2025 Findings.

In this work, we focus on analyzing key patterns that influence the CoT’s performance from both effectiveness and faithfulness perspectives.

Installation

git clone https://github.com/BugMakerzzz/better_cot.git
conda create -n better_cot python=3.10
conda activate better_cot
cd better_cot
pip install -r requirements.txt

Measuring CoT Faithfulness

./script/measure_faith.sh

Measuring Information Flow between Different Components

./script/measure_info.sh

Run the QUIRE Method

./script/llm_reason.sh

If you find our dataset and analysis beneficial, please cite our work:

@misc{li2025betterchainofthoughtreflectioneffectiveness,
      title={Towards Better Chain-of-Thought: A Reflection on Effectiveness and Faithfulness}, 
      author={Jiachun Li and Pengfei Cao and Yubo Chen and Jiexin Xu and Huaijun Li and Xiaojian Jiang and Kang Liu and Jun Zhao},
      year={2025},
      eprint={2405.18915},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2405.18915}, 
}