EPprobit-SN

This repository is associated with the conference proceedings Fasano, Anceschi, Franzolini and Rebaudo (2023a) Efficient expectation propagation for posterior approximation in high-dimensional probit models and Fasano, Anceschi, Franzolini and Rebaudo (2023b) Efficient computation of predictive probabilities in probit models via Expectation Propagation. The key contributions of the two papers are outlined below.

Fasano, Anceschi, Franzolini and Rebaudo (2023a):

[...] we focus on the expectation propagation (EP) approximation of the posterior distribution in Bayesian probit regression under a multivariate Gaussian prior distribution. Adapting more general derivations in Anceschi et al. (2023), we show how to leverage results on the extended multivariate skew-normal distribution to derive an efficient implementation of the EP routine having a per-iteration cost that scales linearly in the number of covariates.

Fasano, Anceschi, Franzolini and Rebaudo (2023b):

[...] we focus on the computation of posterior predictive probabilities in Bayesian probit models via Expectation Propagation (EP). Leveraging more general results in recent literature, we show that such predictive probabilities admit a closed-form expression.

This repository provides codes to replicate the simulation studies reported in Section 4 of Fasano, Anceschi, Franzolini and Rebaudo (2023a) and Section 4 of Fasano, Anceschi, Franzolini and Rebaudo (2023b).

More precisely, we provide the R code to implement Algorithms 1 and 2 presented in Fasano, Anceschi, Franzolini and Rebaudo (2023a) to obtain efficient EP approximations of the posterior moments of the parameters in a probit model with spherical Gaussian prior distribution. We recall that Algorithms 1 and 2 have per-iteration costs O(p²n) and O(pn²), respectively, so the former is used when p<n, the latter otherwise. After the algorithm has converged, one can then efficiently compute the approximated EP posterior predictive probabilities for held-out units exploiting the closed-form expressions presented in Fasano, Anceschi, Franzolini and Rebaudo (2023b).

In addition, we also provide code to perform posterior inference with other three different methods, used for comparison purposes:

1. i.i.d. sampling from the exact unified skew-normal distribution (Durante, 2019)
2. partially factorized mean-field variational Bayes (PFM-VB) approximation (Fasano, Durante and Zanella, 2022)
3. EP implemented via the R function EPprobit from the package EPGLM (Chopin and Ridgway, 2017)

Structure of the repository:

• the functions to implement the above methods can be found in the R source file functions.R
• a tutorial with the code to reproduce the results in the papers is available at Illustration.md