In this lab, you will fine-tune a multi-billion parameter large language model (LLM). We will go through several fundamental concepts of LLMs, including tokenization, templates, and fine-tuning. This lab provides a complete pipeline for fine-tuning a language model to generate responses in a specific style, and you will explore not only language model fine-tuning, but also ways to evaluate the performance of a language model.
You will use Google's Gemma 2B model as the base language model to fine-tune; Liquid AI's LFM-40B as an evaluation "judge" model; and Comet ML's Opik as a framework for streamlined LLM evaluation.