This repository contains code for the paper "Can you rely on your model evaluation? Improving model evaluation with synthetic test data"
For more details, please read our NeurIPS 2023 paper
- Clone the repository
- Create a new conda environment with Python 3.7. e.g:
conda create --name 3s_env python=3.7
- Install requirements in env
pip install -r requirements.txt
- Link the venv to the kernel:
python -m ipykernel install --user --name=3s_env
We highlight different use-cases of 3S-Testing for both subgroup and shift testing in notebooks which can be found in the /use_cases
folder.
If you use this code, please cite the associated paper:
@inproceedings
{3STesting,
title={Can you rely on your model evaluation? Improving model evaluation with synthetic test data},
author={van Breugel, Boris and Seedat, Nabeel and Imrie, Fergus and van der Schaar, Mihaela},
booktitle={Advances in Neural Information Processing Systems},
year={2023}
}