This repository contains HTML code for an interface that helps explore features as learned by a Sparse AutoEncoder (SAE). Please see the interface here. The interface hopes to emulate Anthropic's visualization of features accompanying their Sparse Dictionary Learning paper.
I trained the autoencoder on MLP activations of 1-layer transformer with 512 neurons. The number of features was 4096. 93% of these were alive at the end of training; only 5% of which were ultra-low density neurons. Each feature page contains a feature density histogram, a list of top 10 contexts on which the feature activates, and random samples of activations in various ranges of activation values. Most features are quite interpretable. (There seems to be a correlation between ease of interpretation and bimodality of feature density histograms.)
The codebase used to train the autoencoder is available in another repository here. This interface and the training repository are still a work in progress. For example, Anthropic's visualization covers many more aspects of each feature, such as feature ablations and correlation with neurons. I hope to work on these aspects and also make the code easier to use by others in the future.
Other people have also trained SAEs on language model activations. Please see a (possibly incomplete) list below.