Interpretability

Parameter Decomposition
https://github.com/goodfire-ai/param-decomp

Transformers, LLMs, and the MechInterp Toolbox
https://github.com/ari-holtzman/llms-and-mechinterp

The Neural Geometry Series
https://www.goodfire.ai/research/neural-geometry#

Neuronpedia is an opensource interpretability platform
https://www.neuronpedia.org/

NNSight - Interpret and manipulate the internals of deep learning models
https://github.com/ndif-team/nnsight

Bridging the Attention Gap: Complete Replacement Models for Complete Circuit Tracing
https://interp.open-moss.com/posts/complete-replacement

Zennit is a high-level framework in Python using PyTorch for explaining/exploring neural networks
https://github.com/chr5tphr/zennit

Interpretability of Large Language Models
https://github.com/mega002/llm-interp-tau

Circuit Sparsity Visualizer and Models
https://github.com/openai/circuit_sparsity

ConceptAttention: A method for interpreting multi-modal diffusion transformers
https://github.com/helblazer811/ConceptAttention

Weight-sparse transformers have interpretable circuits
https://cdn.openai.com/pdf/41df8f28-d4ef-43e9-aed2-823f9393e470/circuit-sparsity-paper.pdf