Benchmark Disaggregation Improves Interpretability of Training Dynamics

We include the full code and data for the paper 'Benchmark Disaggregation Improves Interpretability of Training Dynamics'. This repository includes the following directories:

• code: The code used to run the language models on the stimuli.
• stimuli: The preprocessed experimental stimuli in .tsv format and in the format compatible with the language model code.
• results: The output files from the language models. Includes all log-probabilities as surprisals (i.e., negative log-probability).
• stats: The code for generating the plots used in the paper.