🧱 SAEBase: Research Infrastructure for Sparse Autoencoders

🌍 Languages: English | 中文  

Latest software release: v0.6.0 Unified SAE Research Workspace · OpenSAE Playground · Python API

👉 TL;DR. SAEBase helps researchers understand, test, and modify sparse autoencoder (SAE) features in language models. Capture activations, evaluate reconstructions, edit features, measure downstream effects, and reproduce results with checkpoint-aware caches.

🚀 Quick Start   |   🧭 Harness   |   🔬 Workflows   |   🤗 Research Lab   |   🤗 Evaluation Dataset

SAEBase Atlas · Five research trees

SAEBase includes a research Harness for organizing code, scientific knowledge, experiments, computing environments, and agent collaboration. Atlas connects these through five research trees.

Agent Harness: Agent Flow for structured reviews · Cursor rules and skills for coding agents · Experiment tracking for reproducible research.

SAEBase connects pretrained models, SAE dictionaries, activation measurements, controlled feature interventions, benchmarks, and reliability analysis in one reusable workflow.

Capture once, evaluate multiple SAEs. Verified activations can be reused across compatible SAE checkpoints at the same model site, without repeating the language-model forward pass.

🧠 Model and SAE support. Registered checkpoints, named activation sites, compatible adapters and verified model/SAE identities.

🗃️ Reusable activations. Resumable sharded caches, lossless SAE codes, selected-token measurements and integrity checks.

🎛️ Feature interventions. Identity, ablation, scaling, clamping, addition, coalitions and reconstruction, with downstream logit and behavioral readouts.

⚡ Faster experiments. Opt-in suffix replay and KV-cached steering, including live tokenwise SAE edits and vLLM decoder-direction steering.

📊 Benchmarking and reliability. Reconstruction, sparsity and behavioral metrics, paired comparisons, prompt-group bootstrap and negative controls.

🧩 Research integrations. Original SAR and RSS estimators, Input Localization and verified imports of native SAEBench results.

The Research Lab displays public SAE measurements and reports. The table also includes results recorded in separate GPU experiments.

Open the Research Lab · Dataset · Recorded results · Software release notes

Use Python 3.11 or 3.12. The first examples run on CPU without downloading model checkpoints.

git clone https://github.com/OpenSAE/SAEBase.git
cd SAEBase
python -m pip install -e .

python saebase/examples/quickstart/public_api_offline.py
python saebase/examples/quickstart/first_experiment.py --output runs/first-experiment

The first Python example demonstrates activation capture and a feature edit. The second example evaluates two small synthetic SAEs and writes report.html, summary.csv and a reliability report. It prints the exact output paths so you can open the HTML report in a browser.

Install the optional model dependencies, then run the provided Pythia-70M example. This downloads public model and SAE weights.

python -m pip install -e ".[analysis]"
python saebase/examples/interventions/pythia_feature_edit.py

Use the same operations directly in Python:

from saebase.api import (
    load_registered, capture_features, coalition_replacements, evaluate_edit,
)

bundle = load_registered(
    model_key="pythia70m", sae_key="pythia70m_resid_sm", layer=3, device="cpu"
)
tokens = bundle.model.tokens(prompt="Alice Emma Alice")
snapshot = capture_features(bundle.model, bundle.sae, site=bundle.site, tokens=tokens)
features = snapshot.codes.abs().topk(4).indices.tolist()
replacement = coalition_replacements(
    snapshot.codes, selected_indices=features, retained_indices=features[:2]
)
effect = evaluate_edit(
    bundle.model, bundle.sae, site=bundle.site, tokens=tokens, replacements=replacement
)
print(float(effect.logit_delta.float().norm()))

This removes two selected feature contributions and reports the change in next-token logits. For other checkpoints, token selection and editing operations, see the Python API and model registry.

SAEBase separates what gets measured from how the results are analyzed.

Feature edits can preserve the part of an activation not reconstructed by the SAE:

edited_activation = original_activation + decode(edited_codes) - decode(original_codes)

Full SAE reconstruction is a separate experimental control. Reliability estimates resample independent prompt groups rather than treating related token variants as independent observations.

On one RTX 5070 with Qwen3-1.7B Base, opt-in block-suffix replay accelerated the complete 25-sample × 7-condition intervention workload by 1.28×, with unchanged reported metrics. For 128-token generation, live SAE editing with a KV cache achieved 89 tokens/s, and vLLM SAE decoder-direction steering achieved 158 tokens/s. The generation paths use distinct steering protocols.

Full GPU results · Execution details

Verified checkpoints (2026-10-16). Across 18 model-SAE configurations (270M-9B), 48/48 selected model/layer sites passed real-checkpoint compatibility tests, 48/48 completed benchmark evaluations on 64 held-out prompt groups, and 48/48 completed reliability audits. See the verified model and SAE matrix.

Model/SAE matrix · Validation methods

Getting Started · Python API · Examples · Research Lab · Full documentation

Explore the code: SAEBase API · Models · Datasets · Tasks · Harness · Configuration · Repository Guide.

Contributing · Changelog · Citation · Apache-2.0 License · Security

Building open foundations for understanding and evaluating LLMs.
OpenSAE · Issues