Welcome to this curated repository showcasing the fine-tuning of various open-source large language models (LLMs) such as Gemma, LLaMA, Mistral, and others using Hugging Face Transformers, PEFT (LoRA/QLoRA), and other modern libraries.
This repo is designed for researchers, ML engineers, and enthusiasts looking to explore or build on top of custom fine-tuned LLMs.
- β Fine-tuning with Hugging Face Trainer and PEFT (LoRA / QLoRA)
- β Dataset loading and preprocessing
- β Tokenization and model configuration
- β Evaluation with custom metrics
- β Easy-to-edit configs for reproducibility
- β Support for mixed precision (fp16, bf16)
Got improvements, additional models, or tips? Contributions are welcome! Just open an issue or submit a pull request.