CAD

Unofficial re-implementation of Trusting Your Evidence: Hallucinate Less with Context-aware Decoding

Key Features

• Context-Aware Generation: Enhances the relevance of the generated text by incorporating additional context into the decoding process.

Notice

• Currently, this projecit is in a stage where it can only perform simple task generation.
• Available decoding strategy : Greedy Decoding, Top-p Sampling, Top-k Sampling
• Implement based on huggingface transformers' generation_utils.py

Usage Example

cad_model = CAD(model_name="huggyllama/llama-13b", device=0)

contexts = ['Write a quote that ends in the word "early":']
input_texts = ['Better late than']

outputs = cad_model.generate(
                            input_texts=input_texts,
                            use_context_aware=True,
                            contexts=contexts,
                            max_length=20,
                            alpha=0.5,
                            decoding_strategy='top_p',
                            top_p_value=0.9,
                            use_repetition_penalty=True,
                            repetition_penalty_value=1.5,
                            )

print(cad_model.tokenizer.batch_decode(outputs, skip_special_tokens=True)[0])

Citation

@misc{shi2023trusting,
      title={Trusting Your Evidence: Hallucinate Less with Context-aware Decoding}, 
      author={Weijia Shi and Xiaochuang Han and Mike Lewis and Yulia Tsvetkov and Luke Zettlemoyer and Scott Wen-tau Yih},
      year={2023},
      eprint={2305.14739},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}