Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning 🚀

Repo in Progress ...

Official implementation of 📖 Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning

🔗 Blog Post

🔍 Contents

• 🌟 Overview
• 🌟 Method
• 🔥 Dependencies
• 🔥 Setup
• 🔥 Data
• 🔥 Models
• 🚩 Citation

Dependencies

• compatible with python 3.8
• dependencies can be installed using requirements.txt
• The codebase is built around Hugging Face ecosystem and wandb (for monitoring and experiment management).

Install VirtualEnv using the following (optional):

$ [sudo] pip install virtualenv

Create and activate your virtual environment (optional):

$ virtualenv -p python3 venv
$ source venv/bin/activate

Install all the required packages:

$ pip install -r requirements.txt

Data

Data	Reference
GSM8K	📖 , 🗂️
StrategyQA	📖 , 🗂️, 🔗
Causal Understanding	📖, 🗂️, 🔗
OpenBookQA	📖, 🗂️, 🔗
QuaRel	📖, 🗂️, 🔗
QASC	📖, 🗂️, 🔗

Setup

Start by cloning the repository:

git clone git@github.com:debjitpaul/Causal_CoT.git

Citation

@misc{debjit2024frodo,
  title={Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning}, 
  author={Debjit Paul, Robert West, Antoine Bosselut and Boi Faltings}
  year={2024},
  eprint={2402.13950},
  archivePrefix={arXiv},
  primaryClass={cs.CL}}