Add prior_posterior_overlap() for quantifying prior-to-posterior learning

[ishaan-arora-1]

Hi @jgabry — as part of my GSoC proposal work on prior vs posterior comparison plots, I've been prototyping different visualizations and one thing that kept coming up during testing was the need for a simple numeric summary of how much the posterior has moved away from the prior.

MCMCvis has something like this (MCMCtrace with PPO_out = TRUE), but bayesplot doesn't have an equivalent. I think a standalone prior_posterior_overlap() function would be useful even outside the context of the plotting module — it gives users a quick per-parameter number (0 = completely disjoint, 1 = identical) to answer "did my data actually inform this parameter?"

The implementation would be straightforward:

• Accept posterior and prior draws in the standard bayesplot formats (matrix, 3D array, draws objects)
• Support pars, regex_pars, transformations like other mcmc_* functions
• Return a data frame with Parameter and Overlap columns
• Overlap computed as the integral of min(f_prior, f_posterior) over a shared density grid

Would it make sense for me to implement this as a PR? Happy to hear your thoughts on scope or naming.

[jgabry]

Thanks @ishaan-arora-1. I'd be curious to get @avehtari and @florence-bockting's thoughts on this too. Here are a few things that occur to me to get the discussion started:

• If the question we're trying to answer is “how similar are these two marginal distributions?”, I think this overlap calculation seems like a potentially useful summary. But "prior-to-posterior learning" is a lot more complicated. Low "overlap" values could come from shifting locations, contraction, tail shape changes, multimodality, prior-data conflict, etc., so a single number like this doesn't tell those apart, not to mention that it misses anything joint/multivariate. That's not to say it isn't useful, we'd just have to be careful how we describe it if we add it.

• There are many alternative numbers we could use (they all have pros and cons):

  • posterior SD / prior SD (or interval width ratio): this tells us about uncertainty reduction but misses shifts and possibly shape changes
  • (posterior median - prior median) / prior SD: this is good for detecting shifts and prior-data disagreement but doesn't tell us about contraction
  • KL divergence: probably has the strongest theoretical link to "learning" but can be tricky to estimate and explain to users
  • There are some distance metrics that are more appropriate for joint distributions but they're also tricky for various reasons (I'm not an expert in these but maybe @florence-bockting or @avehtari has thoughts)
  • There are probably many more that aren't occurring to me right now

• We could consider comparisons of prior vs posterior predictive distributions, which are more easily interpretable than parameters