Skip to content

Density Ratio Estimation via Infinitesimal Classification (AISTATS 2022 Oral)

Notifications You must be signed in to change notification settings


Repository files navigation

Density Ratio Estimation via Infinitesimal Classification

This repo contains a reference implementation for DRE- as described in the paper:

Density Ratio Estimation via Infinitesimal Classification
Kristy Choi*, Chenlin Meng*, Yang Song, Stefano Ermon
International Conference on Artificial Intelligence and Statistics (AISTATS), 2022. [ORAL]

Note that the code structure is a direct extension of:

Environment setup:

(a) Necessary packages can be found in requirements.txt.

(b) Set the correct Python path using the following command:


(c) Note that we use wandb for keeping track of train/test statistics. You will have to set up wandb in the file.

For the 1-D peaked Gaussian experiments:

To train a time score network with the time-score matching loss only, run:

python3 --toy --config configs/1d_gaussians/time/ --mode=train \
--doc=1d_peaked_gaussians_param_time \

To train a joint score network with both the time and data scores, run:

python3 --toy --config configs/1d_gaussians/joint/ --mode=train \
--doc=1d_peaked_gaussians_param_joint \

For the MI estimation experiments using the joint score matching objective:


python3 --toy  \
  --config configs/gmm_mutual_info/joint/ --mode=train  \
  --doc=mi_40d_param_joint --config.model.type=joint \  \
  --config.seed=7777  \
  --workdir=./results/gmm_mi_40d_param_joint/  \ \

For 80-D, we set,,

For 160-D, we set,, and

For 320-D, we set,,, and

For the MNIST experiments:

First, we use the nsf codebase to train the flow models. All pre-trained model checkpoints (Gaussian, Copula, RQ-NSF) can be found in flow_ckpts/. There is no need to re-train the flow models from scratch and all the time score networks take into account the particular ways that the data has been preprocessed.

(a) For the Gaussian noise model:

python3 --flow \
--config configs/mnist/ \
--mode=train --doc=z_unet_lin_emb_noise \

(b) For the copula:

python3 --flow \
--config configs/mnist/ \
--mode=train --doc=z_unet_lin_emb_copula \ \

(c) For the RQ-NSF flow model:

python3 --flow \
--config configs/mnist/ \
--mode=train --doc=z_unet_lin_emb_flow \ \

PENDING: evaluation code for AIS


If you find this work useful in your research, please consider citing the following paper:

  title={Density Ratio Estimation via Infinitesimal Classification},
  author={Choi, Kristy and Meng, Chenlin and Song, Yang and Ermon, Stefano},
  journal={arXiv preprint arXiv:2111.11010},


Density Ratio Estimation via Infinitesimal Classification (AISTATS 2022 Oral)






No releases published