Model Training

End-to-End Test-Time Training for Long Context
https://github.com/test-time-training/e2e

SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations
https://github.com/Dao-AILab/sonic-moe

An interface library for RL post training with environments
https://github.com/meta-pytorch/OpenEnv

NanoGPT (124M) in 3 minutes
https://github.com/KellerJordan/modded-nanogpt

We Got Claude to Fine-Tune an Open Source LLM
https://huggingface.co/blog/hf-skills-training

Evolution Strategies at the Hyperscale
https://github.com/ESHyperscale/HyperscaleES

RLinf: Reinforcement Learning Infrastructure for Agentic AI
https://github.com/RLinf/RLinf

The Smol Training Playbook
https://huggingface.co/spaces/HuggingFaceTB/smol-training-playbook

Post-training 101
https://tokens-for-thoughts.notion.site/post-training-101

SkyRL tx: An open source project to implement the Tinker API
https://novasky-ai.notion.site/skyrl-tx

Tunix: A JAX-Native Library for LLM Post-Training
https://developers.googleblog.com/en/introducing-tunix-a-jax-native-library-for-llm-post-training/

torchtitan - A PyTorch native platform for training generative AI models
https://github.com/pytorch/torchtitan

The Ultra-Scale Playbook: Training LLMs on GPU Clusters
https://huggingface.co/nanotron

How to Scale Your Model
https://jax-ml.github.io/scaling-book/index