This repo contains the code of the following 9 papers:

Algebraic Structure of Representation in Reasoning tasks

Provable Scaling Laws of Feature Emergence from Learning Dynamics of Grokking

Yuandong Tian

arXiv'25

Composing Global Solutions to Reasoning Tasks via Algebraic Objects in Neural Nets

Yuandong Tian

NeurIPS'25

Analysis of Transformer training dynamics (./ssl/real-dataset)

Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer

Yuandong Tian, Yiping Wang, Beidi Chen, Simon Du

NeurIPS'23

JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention

Yuandong Tian, Yiping Wang, Zhenyu Zhang, Beidi Chen, Simon Du

ICLR'24

Analysis of Self-supervised learning (./ssl)

Understanding the Role of Nonlinearity in Training Dynamics of Contrastive Learning

Yuandong Tian

ICLR'23

Understanding Deep Contrastive Learning via Coordinate-wise Optimization

Yuandong Tian

NeurIPS'22 Oral

Understanding self-supervised Learning Dynamics without Contrastive Pairs

Yuandong Tian, Xinlei Chen, Surya Ganguli

ICML 2021 link, Outstanding Paper Honorable Mention

Understanding Self-supervised Learning with Dual Deep Networks

Yuandong Tian, Lantao Yu, Xinlei Chen, Surya Ganguli

arXiv link

Teacher-student setting in supervised learning

Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension (./student_specialization)

Yuandong Tian

ICML 2020 link

Luck Matters: Luck Matters: Understanding Training Dynamics of Deep ReLU Networks (./luckmatter)

Yuandong Tian, Tina Jiang, Qucheng Gong, Ari Morcos

arxiv link