This project demonstrates how to fine-tune the Gemma-2B language model for conversation summarization using QLoRA (Quantized Low-Rank Adaptation) on the SAMSum dataset. It leverages Hugging Face Transformers, PEFT, and BitsAndBytes libraries for efficient and effective fine-tuning.
The goal of this project is to create a model capable of generating concise and accurate summaries of conversations. We achieve this by fine-tuning a pre-trained Gemma-2B model using QLoRA, a technique that reduces memory footprint and speeds up training while maintaining performance. The SAMSum dataset, consisting of dialogues and their corresponding summaries, is used for training and evaluation.
- Fine-tuning Gemma-2B: Leverages the powerful Gemma-2B language model as the foundation.
- QLoRA for Efficiency: Employs Quantized Low-Rank Adaptation for memory-efficient fine-tuning.
- SAMSum Dataset: Utilizes a well-established dataset for conversation summarization.
- Hugging Face Ecosystem: Integrates seamlessly with Hugging Face Transformers, PEFT, and Datasets libraries.
- Detailed Code: Provides clear and well-commented code for easy understanding and reproducibility.
-
Clone the repository:
-
Install dependencies:
-
Authenticate with Hugging Face Hub:
-
Run the Colab notebook: Open the notebook in Google Colab and execute the cells. This will download the dataset, fine-tune the model, and save the results.
After fine-tuning, you can use the model to generate summaries of conversations:
The fine-tuned model achieves promising results on the SAMSum dataset, demonstrating its ability to generate informative and concise summaries.
Contributions are welcome! If you find any issues or have suggestions for improvements, please feel free to open an issue or submit a pull request.