in this repo I train my skills in ML by practicing implementing experiments in jax (from scratch, except the backwards pass), and optimizing them
Jax is (basically) numpy but with autograd. This lets me focus on getting good at implementation/training details without tediously implementing backprop by hand
I am doing this in a community on X called "LEARN BY BUILDING" where we build projects as a way to learn tech stuff: programming, robotics, etc. If you have something you want to learn, join, come up with a project, and post whenever you make progress!
Use GRPO to survive a 3-body physics simulation with rockets (status: env implemented, GRPO implemented, but training needs refining)
Double deep Q net that plays 6D snake (status: 2D version completed, needs opengl renderer, needs upgrade from regular DQN to double DQN)
Pixtral inference/LoRA training from scratch in jax
Train a small (19.2M) transformer on the dictionary
Train an RNN/LSTM on my x posts
- custom CNN with skip connections on MNIST
Train an MLP on MNIST
become very fast at replicating papers and complex experiments/architectures milestones:
- replicate 1 paper per 2 months
- replcate 1 paper every month
- replicate 1 paper every 2 weeks
- replicate 1 paper every week
- replicate 2 papers a week
- 4
- 7
- replicate 2 papers a day
(here, 'paper' means a model, or a custom experiment, or a random paper. this is not a super concrete metric bc of differences in paper difficulties, but it suffices)