Example notebooks in R using rstanarm, rstan, bayesplot, loo, projpred.
- Use of reference models in variable selection at Laplace's demon seminar series
- Model assessment, comparison and selection at Master class in Bayesian statistics, CIRM, Marseille
- Model assessment and model selection aka Basics of cross-validation tutorial at StanCon 2018 Helsinki
- Regularized horseshoe talk at StanCon 2018 Asilomar
- Model selection tutorial at StanCon 2018 Asilomar
- Basics of predictive performance estimation
- When cross-validation is not needed
- Simple model we trust - betablockers
- When cross-validation is useful
- On accuracy of cross-validation
- Cross-validation and hierarchical models
- When cross-validation is not enough
- large number of models - diabetes
- loo 2.0
- Projection predictive model selection
- loo 2.0
- Projection predictive model selection (projpred) examples
- collinearity - mesquite
- random data vs original data - candy
- winequality-red
- bodyfat
- See also projpred quick start vignette
- LOO-R^2
- When LOO and other cross-validation approaches are valid
- Parsimonious principle vs integration over all uncertainties
- Comments on Limitations of Bayesian Leave-One-Out Cross-Validation for Model Selection
- I am the supercargo
- Nice horseshoe example in Bayes Sparse Regression case study by Michael Betancourt
- Afrabandpey, H., Peltola, T., Piironen, J., Vehtari, A., and Kaski, S. (2019). Making Bayesian predictive models interpretable: A decision theoretic approach. arXiv preprint arXiv:1910.09358
- Bürkner, P.-C., Gabry, J., Vehtari, A. (2018). Leave-one-out cross-validation for non-factorizable normal models. arXiv:1810.10559
- Bürkner, P.-C., Gabry, J., Vehtari, A. (2020). Approximate leave-future-out cross-validation for time series models. Journal of Statistical Computation and Simulation, doi:10.1080/00949655.2020.1783262. Online. Preprint arXiv:1902.06281
- Gelman, A., Hwang, J., and Vehtari, A. (2014). Understanding predictive information criteria for Bayesian models. Statistics and Computing, 24(6):997–1016. Preprint
- Gelman, A., Goodrich, B., Gabry, J., and Vehtari, A. (2018). R-squared for Bayesian regression models. The American Statistician, doi:10.1080/00031305.2018.1549100. Online.
- Magnusson, M., Andersen, M.R., Jonasson, J., Vehtari, A. (2019). Bayesian leave-one-out cross-validation for large data. Thirty-sixth International Conference on Machine Learning, PMLR 97:4244--4253. Online.
-
- Magnusson, M., Andersen, M.R., Jonasson, J., Vehtari, A. (2020). Leave-one-out cross-validation for Bayesian model comparison in large data. Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 108:341-351. Online. preprint arXiv:2001.00980.
- Paananen, T., Piironen, J., Bürkner, P.-C., and Vehtari, A. (2020). Implicitly adaptive importance sampling. arXiv:1906.08850
- Pavone, F., Piironen, J., Bürkner, P.-C., and Vehtari, A- (2020). Using reference models in variable selection. arXiv preprint arXiv:2004.13118
- Piironen, J. and Vehtari, A. (2016), Comparison of Bayesian predictive methods for model selection, Statistics and Computing 27(3), 711–735. Online
- Piironen, J., and Vehtari, A. (2017). On the hyperprior choice for the global shrinkage parameter in the horseshoe prior. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, PMLR 54:905-913. Online
- Piironen, J., and Vehtari, A. (2017). Sparsity information and regularization in the horseshoe and other shrinkage priors. In Electronic Journal of Statistics, 11(2):5018-5051. Online
- Piironen, J., and Vehtari, A. (2018). Iterative supervised principal components. Proceedings of the 21th International Conference on Artificial Intelligence and Statistics, accepted for publication. arXiv preprint arXiv:1710.06229
- Piironen, J., Paasiniemi, M., and Vehtari, A. (2020). Projective Inference in High-dimensional Problems: Prediction and Feature Selection. Electronic Journal of Statistics, 14(1):2155-2197. Online. Preprint arXiv:1810.02406
- Vehtari, A., Gelman, A., Gabry, J. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing. 27(5):1413–1432. arXiv preprint.
- Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2019). Pareto smoothed importance sampling. arXiv preprint.
- Vehtari, A., Mononen, T., Tolvanen, V., and Winther, O. (2016). Bayesian leave-one-out cross-validation approximations for Gaussian latent variable models. JMLR, 17(103):1–38. Online
- Vehtari, A. and Ojanen, J.: 2012, A survey of Bayesian predictive methods for model assessment, selection and comparison, Statistics Surveys 6, 142–228. Online
- Williams, D. R., Piironen, J., Vehtari, A., and Rast, P. (2018). Bayesian estimation of Gaussian graphical models with projection predictive selection. arXiv:1801.05725
- Yao, Y., Vehtari, A., Simpson, D., and Gelman, A. (2017). Using stacking to average Bayesian predictive distributions. In Bayesian Analysis, doi:10.1214/17-BA1091, Online