This starter notebook is provided by the Keras team.
In this competition, we aim is to build AI models that can solve tough math problems, in other words, creating LLM models capable of solving Math Olympiad problems. This notebook will guide you through the process of fine-tuning the Gemma LLM model with LoRA to solve math problems using KerasNLP. With KerasNLP, fine-tuning with LoRA becomes straightforward with just a few lines of code.
Did you know:: This notebook is backend-agnostic? Which means it supports TensorFlow, PyTorch, and JAX backends. However, the best performance can be achieved with JAX. KerasNLP and Keras enable the choice of preferred backend. Explore further details on Keras.
Note: For a deeper understanding of KerasNLP, refer to the KerasNLP guides.
You can find the notebook in notebooks/ or you can access the notebook in kaggle here.