LangFIR: Discovering Sparse Language-Specific Features from Monolingual Data for Language Steering

Code for reproducing the experiments in the paper.

Setup

Requires Python 3.12 and a CUDA-capable GPU (~48GB VRAM), or remote inference via NDIF.

pip install -r requirements.txt

Repository Structure

steering_experiment/

Comparison of 8 steering vector construction methods on two tasks.

• translation/ — Sentence-translation task
• xquad/ — Cross-lingual answer generation on XQuAD (5 languages; steering vectors are built from FLORES+ dev)

Each folder follows the same pattern:

• data_construction_*.ipynb — Construct steering vectors for each method (run first)
• interventions_*.ipynb — Apply steering vectors and evaluate (language detection accuracy, BLEU)

Validation interventions have separate notebooks for Gemma and Llama; test interventions use a single notebook.

ablation/

Ablation studies isolating the contribution of individual design choices.

• data_construction_ablation.ipynb — Construct ablated steering vector variants
• interventions_ablation.ipynb — Evaluate ablated variants

analyses/

• LangFIR_analyses.ipynb — Analysis of the feature identification pipeline (filtering steps, feature properties)
• directional_ablation.ipynb — Directional ablation measuring the effect of removing identified feature directions on cross-entropy loss

Workflow

Run data construction notebooks before their corresponding intervention notebooks. Analysis notebooks are independent.