NanoLens v1.0 — Initial Release

NanoLens v1.0

First release of NanoLens — a mechanistic interpretability toolkit built
from scratch on a 25M parameter character-level transformer trained on
Dostoevsky.

What's included

• Pretrained checkpoint: 8 layers, 8 heads, 512 embedding dim
• Trained for 5000 steps on 4.4M characters (Crime and Punishment,
  Brothers Karamazov, The Idiot)
• Final val loss: 1.1144 | Train/val gap: 0.053

Checkpoint

• nanolens_v1.pt — ~111MB, drop into checkpoints/ directory

Research findings in this release

• 6 documented attention head types across 64 heads
• Previous token heads, BOS sink heads, word boundary heads,
  abstract semantic routing heads confirmed
• Early layers (0-2) encode local sequential structure
• Late layers (5-7) perform abstract semantic routing

Usage

See README for full quickstart. Requires Python 3.10+, PyTorch 2.0+.

python cli.py --checkpoint checkpoints/nanolens_v1.pt
python cli.py --checkpoint checkpoints/nanolens_v1.pt --inspect "Raskolnikov hesitated"

Label

None — this is a stable research release, not a pre-release.