A sketch of a Transformer in Rust for pedagogical purposes.
This code is the companion piece for the accompanying blog post.
This repository contains an implementation of most of a decoder-only Transformer language model, implemented in Rust. The code is not runnable or designed to be run, but exists as a tool for explaining how these models work, and some of the perspectives employed or developed by the Interpretability team at Anthropic in our work on these models.
transformer-rs - a pedagogical sketch of a Transformer model
Written in 2022 by Nelson Elhage nelhage@nelhage.com
To the extent possible under law, the author(s) have dedicated all copyright and related and neighboring rights to this software to the public domain worldwide. This software is distributed without any warranty.
You should have received a copy of the CC0 Public Domain Dedication along with this software. If not, see http://creativecommons.org/publicdomain/zero/1.0/.