pip install transcoders-slimLoad pre-trained transcoders from Jacob Dunefsky's repository: https://github.com/jacobdunefsky/transcoder_circuits
from transcoders.transcoder import Transcoder
from transcoders_slim.load_pretrained import load_pretrained
transcoders: dict[str, Transcoder] = load_pretrained()
for name, transcoder in transcoders.items():
print(name, transcoder)Run a transcoder:
d_in = transcoder.d_in
d_tr = transcoder.d_sae
seq_len = 32
tr_in = torch.zeros(1, seq_len, d_in).to(transcoder.device)
tr_out, tr_hid = transcoder(tr_in)[:2]
Refer to Setup for how to set up development environment.