LLM Hallucinantions

Code for reproducing the experiments in Unfamiliar Finetuning Examples Control How Language Models Hallucinate.

Setup

pip install torch --extra-index-url https://download.pytorch.org/whl/cu118
pip install -e .

Install FActScore according to instructions here. Replace “InstructGPT” with “ChatGPT” in line 26 of factscore/atomic_facts.py in your FActScore installation, since text-davinci-003 is no longer supported.

Download the training data and our finetuned checkpoints here. Place the downloaded folder inside llm_hallucinations/examples.

Usage

See llm_hallucinations/examples for training, evaluation, and plotting code.

Acknowledgements

Our codebase was built on top of trlx.