Repository navigation
airgpt: a nanochat-style pipeline for Apple Silicon in MLX #863
navthings
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
i built a small mac version of the nanochat idea for people without an nvidia gpu (me included). it's in mlx, does tokenizer training, pretraining, chat sft and chat in one command, and runs off a time budget instead of steps so presets take the same wall-clock time on any mac.
on an m5 macbook air the one hour preset gets an 8M model to val loss 2.27 on tinystories + cosmopedia. it's a lot smaller than nanochat's runs but it's been a fun way to see the whole thing work on a laptop.
https://github.com/navthings/airgpt
thanks for nanochat, a lot of the structure comes from reading it.
All reactions