Repository navigation
This release provides the task snapshots, observed learning trajectories, and function vectors for What Do Language Models Learn and When? The Implicit Curriculum Hypothesis (COLM 2026).
- 88 tasks and 19,863 archived reference examples.
- 38,889 observed accuracy measurements across 10 models.
- 806 function vectors in 10 model ZIPs.
Browse or download on Hugging Face. For a complete local copy, download and extract elementaltask-v0.1.tar.gz, then run python load_artifacts.py --root . --verify inside the extracted directory. The individual task, trajectory, and per-model FV ZIPs are also attached. RELEASE_ASSET_SHA256SUMS.txt checks the downloadable assets; SHA256SUMS checks the full extracted snapshot.
Some third-party task materials are omitted from the artifact release pending redistribution permission. See the coverage manifest for included tasks. Original project data and artifact documentation use CC BY 4.0; code and the attributed IOI examples retain MIT terms. A BibTeX entry is included.
All 44 OLMo-2 13B task scores in the expanded camera-ready experiment reproduce to numerical precision using the complete local research inputs. Older 26-task predictions retain preprocessing differences; the public subset omits tasks needed for a complete prediction basis. Other models received artifact integrity checks, not a full experiment replay. OLMo-2 13B arrays are full-residual representations, as recorded in their extraction metadata.
The source archive for this artifact tag contains the release documentation and manifests. Use the attached archives or Hugging Face snapshot for the data.