Optimizing LLMs with Fine-Tuning and Prompt Engineering

This repository contains code for the Optimizing LLMs with Fine-Tuning and Prompt Engineering

Notebooks

Comparing Fine-tuned OpenAI + BERT

• bert_app_review.ipynb: Fine-tuning a BERT model for app review classification.
• openai_app_review_fine_tuning.ipynb: Fine-tuning OpenAI models for app review classification.

Fine-tuning embeddings

• Fine-tuning Embeddings For Rec Engines: Fine-tuning embedding engines using custom preference data

• Fine-tuning Embeddings with Synthetic Data - Using GPT-4o to create synthetic queries for a corpus to increase the quality of open-source embedding models

Working with FLAN-T5 models using Reinforcement Learning

• Colab Version:

SAWYER - Training a chat model with RLF

• SAWYER_LLAMA_SFT.ipynb: Fine-tuning the Llama-3 model to create the SAWYER bot.
  • colab version
• SAWYER_Reward_Model.ipynb: Training a reward model from human preferences for the SAWYER bot.
  • colab version
• SAWYER_RLF.ipynb: Applying Reinforcement Learning from Human Feedback (RLHF) to align the SAWYER bot.
  • colab version
• SAWYER_USE_SAWYER.ipynb: Using the SAWYER bot.
  • colab version
• Adapting SAWYER to know more about ML

Distillation + Quantization

• distillation_example_1.ipynb: Exploring knowledge distillation techniques for transformer models.
• distillation_example_2.ipynb: Advanced distillation methods and applications.
• llama_quantization.ipynb: Quantizing Llama models for efficient deployment.
• Llama.cpp - Using LLMs with llama.cpp