-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add WAIC and PSIS-LOO #19
Comments
Hi Carl, These both look useful to add, so will try and get on them when I have time. Have my PhD confirmation coming up in a few weeks, so won't be a lightning fast addition though, so if you or your group want it in asap please feel free to make a pull request. Cheers |
It is not urgent (at least for me). My feeling was, it should be added as comparison to AIC/BIC/DIC. Recently I found an pymc3 implementation here: https://github.com/pymc-devs/pymc3/blob/master/pymc3/stats.py Good luck for your PhD. |
Ah yeah, thats a nice compact implementation that I can credit over,
cheers. Out of curiosity, what group do you work with?
…On Wed, Apr 5, 2017 at 10:12 PM, carlkl ***@***.***> wrote:
It is not urgent (at least for me). My feeling was, it should be added as
comparison to AIC/BIC/DIC. Recently I found an pymc3 implementation here:
https://github.com/pymc-devs/pymc3/blob/master/pymc3/stats.py
Good luck for your PhD.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#19 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AGb5Wd3OzW9qSV6zv16pMWtrUsqJgQw8ks5rs4UWgaJpZM4M0GsS>
.
|
Your Q: accelerator physics. BTW: bayesian analysis ist just a hobby project for me, for prokrastination and self-learning. |
Ah fair enough. And thats interesting, didn't realise ChainConsumer had
done outside of the Dark Energy Survey yet!
…On Wed, Apr 5, 2017 at 10:31 PM, carlkl ***@***.***> wrote:
Your Q: accelerator physics.
BTW: bayesian analysis ist just a hobby project for me, for
prokrastination and self-learning.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#19 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AGb5WViyWEWaiGPKNCCPFGPn7Xb7kQUWks5rs4mrgaJpZM4M0GsS>
.
|
BTW: There is a nice comparsion (in german): https://epub.ub.uni-muenchen.de/31977/1/BA_Erdemir.pdf with an easy to understand explanation why WAIC/WBIC should be preferred over AIC/BIC. |
Excellent, time to brush up on meine Deutsch. Danke!
…On Wed, Apr 5, 2017 at 10:44 PM, carlkl ***@***.***> wrote:
BTW: There is a nice comparsion (in german): https://epub.ub.uni-muenchen.
de/31977/1/BA_Erdemir.pdf with an easy to understand explanation why
WAIC/WBIC should be preferred over AIC/BIC.
I didn't get the point with the original WAIC papers.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#19 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AGb5WT-tH6Nye1gg6ozNZ8IiGeySTbi9ks5rs4yWgaJpZM4M0GsS>
.
|
Closing this as an old issue I never got the time to work on, sorry Carl. |
PSIS-LOO
Pareto smoothed importance sampling. Aki Vehtari, Andrew Gelman and Jonah Gabry (2016).
https://arxiv.org/abs/1507.04544
see also https://github.com/avehtari/PSIS
Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Aki Vehtari, Andrew Gelman, Jonah Gabry 29 June 2016
http://www.stat.columbia.edu/~gelman/research/unpublished/loo_stan.pdf
WAIC
http://watanabe-www.math.dis.titech.ac.jp/users/swatanab/waicwbic_e.html
see also http://www.stat.columbia.edu/~gelman/research/published/waic_understand3.pdf
Both methods use the MCMC chain for evaluation.
The text was updated successfully, but these errors were encountered: