Feature Request: Automatic Differentiation Variational Inference (ADVI)

[matthew-mcateer]

I've seen both MCMC and NUTS being initialized with a mean taken from ADVI. This was at one point a feature in Pymc. Thus far I have not seen anything resembling this in either the tf.vi section, or tensorflow_probability as a whole. It would be an awesome feature.

[jvdillon]

Hi Matthew. Thanks for your comment. This is a very good idea. Im currently spec-ing out how to make these hybridizations more ergonomic. Ill share my design doc with you after discussing it with my team.

[srvasude]

I'm going to close this bug. ADVI exists in a form insomuchas we have fit_surrogate_posterior which allows fitting a surrogate to a unnormalized density: https://github.com/tensorflow/probability/blob/main/tensorflow_probability/python/vi/optimization.py#L29

https://www.tensorflow.org/probability/api_docs/python/tfp/experimental/vi/build_factored_surrogate_posterior

lets you create your surrogate model in arbitrary ways.