Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift

by Christina Baek, Yiding Jiang, Aditi Raghunathan, and Zico Kolter from Carnegie Mellon University.

This is the repository for our paper https://arxiv.org/abs/2206.13089. We provide the model predictions and the code used to generate Figure 4 (For experimental details, see Section 5.3 Estimating performance along a training trajectory) in the main body of our paper. The code for our algorithm is in the aline method in agreement_trajectory.ipynb.

Prerequisites

• Python
• matplotlib
• pandas
• numpy
• statsmodels
• scipy