Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Implement a reasonable Griddy Gibbs #642
The basic idea called "Griddy Gibbs" is that one can (approximately) Gibbs-sample from any unnormalized 1-dimensional continuous posterior density by just doing the quadrature.
Somehow this idea has gotten metamorphosed (at least for hyperparameter inference in our crosscat implementations) into the workable but rather unsatisfying approximation "Approximate the hyper-prior as a discrete distribution on a grid of values, and do enumerative Gibbs on that."
The original Ritter, Tanner 1991 paper "The Griddy Gibbs Sampler" actually alludes to the possibility of doing one step better than that, namely using one's favorite quadrature technique to form an approximate cumulative distribution function (which, notably, need not be a step function) and sampling from that.
In fact, we can do better still: treat the sample from said approximate CDF as an M-H proposal (independent of the current state), and include the possibility of rejecting it. For approximate CDFs that are continuous (e.g., piecewise linear), the acceptance ratio is both well-defined and easy to compute. This variant stops being a Gibbs sampler, strictly speaking, but gains the benefit of being a transition operator with exactly the right invariant distribution. The quality of the quadrature approximation will determine the acceptance ratio (and, in fact, one can imagine auto-tuning the choice of grid fineness in order to reach good acceptance rates).
Why do this?