Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Feature/issue 2814 warmup auto #2815
The goals/reasons are outlined here: #2814
There'll be a CmdStan pull to go with this (and this probably shouldn't go in until that pull is good too).
This adds another metric that the samplers use to compute all their gradients and such: src/stan/mcmc/hmc/hamiltonians/auto_e_metric.hpp, and src/stan/mcmc/hmc/hamiltonians/auto_e_point.hpp
And then an adaptation routine to actually compute that metric: src/stan/mcmc/auto_adaptation.hpp
Edit: To review this pull request you'll want to pull this version of cmdstan and at least try out the adaptation on a couple models: stan-dev/cmdstan#729
The adaptation routine updates the metric and tells it whether to act like a dense or diagonal metric at the end of each warmup stage where the metric is recomputed.
How to Verify
The new tests can be run with:
Yet to be written
Copyright and Licensing
Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company): Columbia University
By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses:
Before consideration this is going to require a good bit of empirical validation more than what's in the arXiv paper, especially with regard to varying dimensions and curvatures. To be open I am hesitant about the robustness of automatically switching between diagonal and dense given how small the early windows are, and how noisy those off-diagonal estimates are (not to mention the eigenvalue approximations). For something like this goes to go in it will have to be verified to work properly for diagonally-dominant problems, dense-dominated problems, and everything in between without a significant increase in cost.
Keep in mind that we're trying to minimize the sampler variants in the code base and not have "experimental" versions on dev/master. At some point we'll clean up the other samplers in there.
Yup, hopefully we can find some models that break and learn stuff!
In general I recommend working through the validation before creating a pull request and using up testing resources. Instead a branch can be discussed on Discourse.
Because of how this proposal modifies warmup it will require studying, at the very least,