A Model of Understanding in Deep Learning Systems: code and notebooks

This repository contains a small set of runnable notebooks that replicate the key computational examples from the accompanying paper.

Notebooks

Open the notebooks in order:

1. notebooks/01_torus_structural.ipynb
   ReLU implicit field regression for a torus, isosurface extraction via Marching Cubes, and basic mesh topology diagnostics (Euler characteristic and genus).

2. notebooks/02_kepler_orbits.ipynb
   Baseline MLP fit versus a compact geometric "KeplerNet" model on historical Mars data. (Requires two CSV files in data/, see data/README.md.)

3. notebooks/03_modular_addition_grokking.ipynb
   Tiny Transformer for modular addition, grokking dynamics, and a Fourier analysis of the learned logits.

4. notebooks/04_othello_world_model.ipynb
   A minimal Othello-GPT style model trained on random games, plus linear probes for board state.

5. notebooks/05_spline_landscape.ipynb
   A small ReLU network learning an implicit 2.5D landscape surface, used to illustrate the spline (piecewise-linear) interpretation.

Installation (local)

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Then launch Jupyter:

jupyter lab

Running in Colab

Each notebook includes a small helper that installs missing packages automatically when run in Colab.

Outputs

All generated figures, checkpoints, and cached datasets are written under artifacts/ and are ignored by git.