Multiple Model Comparison Test.
This repository contains resources for finding the best model with multiple model comparison. The algorithm is described in our paper,
Kernel Stein Tests for Multiple Model Comparison Jen Ning Lim, Makoto Yamada, Bernhard Schölkopf, Wittawat Jitkrittum NeurIPS 2019
How to install?
The package can be installed
pip install -e /path/to/the/folder/of/this/repo/after/clone
pip install git+https://github.com/jenninglim/model-comparison-test.git
Once installed, you should be able to do
import reltest without any error.
The current implementation relies on an accurate approximation of the inverse CDF in the tail regions of the truncated normal (see 6). The implementation uses the inverse CDF to calculate the rejection threshold. If the test statistic is greater than the rejection threshold, we reject the null hypothesis.
The p-values produced from the modules may not be accurate. At the time of
writting this code there were several problems with
scipy.stats.truncnorm module. See
4. This may have been fixed in
the pull request 5 in SciPy 1.4.0.