(GitHub issue/PR number in parentheses)
compare()is now deprecated in favor of
Use markdown syntax in roxygen documentation wherever possible. (#108)
(GitHub issue/PR number in parentheses)
loo_compare()for model comparison that will eventually replace the existing
New vignette on LOO for non-factorizable joint Gaussian models. (#75)
New vignette on "leave-future-out" cross-validation for time series models. (#90)
New glossary page (use
help("loo-glossary")) with definitions of key terms. (#81)
se_diffcolumn in model comparison results. (#78)
Improved stability of
log_ratiosare very small. (#74)
r_eff=NAto suppress warning when specifying
r_effis not applicable (i.e., draws not from MCMC). (#72)
Update effective sample size calculations to match RStan's version. (#85)
Naming of k-fold helper functions now matches scikit-learn. (#96)
This is a major release with many changes. Whenever possible we have opted to deprecate rather than remove old functionality, but it is possible that old code that accesses elements inside loo objects by position rather than name may error.
New package documentation website http://mc-stan.org/loo/ with vignettes, function reference, news.
Updated existing vignette and added two new vignettes demonstrating how to use the package.
psislw()(now deprecated). This version implements the improvements to the PSIS algorithm described in the latest version of https://arxiv.org/abs/1507.02646. Additional diagnostic information is now also provided, including PSIS effective sample sizes.
weights()method for extracting smoothed weights from a
normalizecontrol whether the weights are returned on the log scale and whether they are normalized.
Updated the interface for the
loo()methods to integrate nicely with the new PSIS algorithm. Methods for log-likelihood arrays, matrices, and functions are provided. Several arguments have changed, particularly for the
loo.functionmethod. The documentation at
help("loo")has been updated to describe the new behavior.
The structure of the objects returned by the
loo()function has also changed slightly, as described in the Value section at
help("loo", package = "loo").
loo_model_weights()computes weights for model averaging as described in https://arxiv.org/abs/1704.02030. Implemented methods include stacking of predictive distributions, pseudo-BMA weighting or pseudo-BMA+ weighting with the Bayesian bootstrap.
options(loo.cores=...)is now deprecated in favor of
options(mc.cores=...). For now, if both the
mc.coresoptions have been set, preference will be given to
loo.coresuntil it is removed in a future release. (thanks to @cfhammill)
example_loglik_matrix()that provide objects to use in examples and tests.
When comparing more than two models with
compare(), the first column of the output is now the
elpddifference from the model in the first row.
New helper functions for splitting observations for K-fold CV:
kfold_split_stratified(). Additional helper functions for implementing K-fold CV will be included in future releases.
- Introduce the
E_loofunction for computing weighted expectations (means, variances, quantiles).
pareto_k_idsconvenience functions for quickly identifying problematic observations
- pareto k values now grouped into
(1, Inf)(didn't used to include 0.7)
- warning messages are now issued by
print.looshows a table of pareto k estimates (if any k > 0.7)
- Add argument to
compareto allow loo objects to be provided in a list rather than in
- Update references to point to published paper
- GitHub repository moved from @jgabry to @stan-dev
- Better error messages from
- Fix example code in vignette (thanks to GitHub user @krz)
- Add warnings if any p_waic estimates are greather than 0.4
- Improve line coverage of tests to 100%
- Update references in documentation
- Remove model weights from
In previous versions of loo model weights were also reported by
compare. We have removed the weights because they were based only on the point estimate of the elpd values ignoring the uncertainty. We are currently working on something similar to these weights that also accounts for uncertainty, which will be included in future versions of loo.
This update makes it easier for other package authors using loo to write
tests that involve running the
loo function. It also includes minor bug
fixes and additional unit tests. Highlights:
- Don't call functions from parallel package if
- Return entire vector/matrix of smoothed weights rather than a summary statistic when
psislwfunction is called in an interactive session.
- Test coverage > 80%
This update provides several important improvements, most notably an alternative method for specifying the pointwise log-likelihood that reduces memory usage and allows for loo to be used with larger datasets. This update also makes it easier to to incorporate loo's functionality into other packages.
- Add Ben Goodrich as contributor
- S3 generics and
functionmethods for both
waic. The matrix method provide the same functionality as in previous versions of loo (taking a log-likelihood matrix as the input). The function method allows the user to provide a function for computing the log-likelihood from the data and posterior draws (which are also provided by the user). The function method is less memory intensive and should make it possible to use loo for models fit to larger amounts of data than before.
label_pointsargument, which, if
TRUE, will label any Pareto
kpoints greater than 1/2 by the index number of the corresponding observation. The plot method also now warns about
kthat are not shown in the plot.
comparenow returns model weights and accepts more than two inputs.
- Allow setting number of cores using
options(loo.cores = NUMBER).
- Updates names in package to reflect name changes in the accompanying paper.
- Better handling of special cases
loo_and_waicfunction in favor of separate functions
- Initial release