This project is part of the LibreLLM initiative.
This repository defines the base architecture of LibreLLM models, including configuration, training loop, optimizations (FlashAttention, RoPE, GQA, 1bit), and optional support for tool calling and multimodal input.