Feature/bbvb family

[dustinvtran]

Summary:

Allows one to specify an arbitrary class of variational distribution in order to perform variational inference with. I built a base class base_family.hpp from which all variational distributions should inherit from and build methods for, c.f., variational/families directory, which makes the two Gaussian families we currently have as special cases.

Intended Effect:

No effect on the user/outputs. Removes a ton of hard coded nastiness and makes it easier to do black box variational inference with whatever desired variational distributions.

How to Verify:

Run tests, run models. Fix a seed and check that the estimates are exactly the same as in the original codebase.

Side Effects:

None

Documentation:

None

Reviewer Suggestions:

@akucukelbir

[dustinvtran]

I think this is a good enough separation of the raw inference algorithm from the variational families used, i.e., the raw inference algorithm simply calls upon generic methods for a variational family such as variational.sample() and variational.mean().

Ideally the gradient of the ELBO should exist as a function inside advi.hpp and not in the variational family classes. At the moment it's so directly tied to the variational families that for now I'm leaving it as variational.calc_grad().

[stan-buildbot]

Build Log
last 20 lines

[...truncated 21 lines...]
Starting build job Stan Pull Request - Tests - Header Math.
Starting build job Stan Pull Request - Tests - Header.
Starting build job Stan Pull Request - Windows - Tests - Unit.
Starting build job Stan Pull Request - Tests - Unit.
Starting build job Stan Pull Request - Tests - Integration.
Finished Build : #426 of Job : Stan Pull Request - Tests - Header with status : SUCCESS
Finished Build : #32 of Job : Stan Pull Request - Tests - Header Math with status : SUCCESS
Finished Build : #275 of Job : Stan Pull Request - Windows - Tests - Unit with status : SUCCESS
Finished Build : #33 of Job : Stan Pull Request - Tests - Unit Math with status : SUCCESS
Finished Build : #428 of Job : Stan Pull Request - Tests - Integration with status : SUCCESS
Finished Build : #428 of Job : Stan Pull Request - Tests - Unit with status : SUCCESS
Starting build job Stan Pull Request - CppLint.
Starting build job Stan Pull Request - Math Dependencies.
Starting build job Stan Pull Request - CppLint Math.
Finished Build : #134 of Job : Stan Pull Request - Math Dependencies with status : SUCCESS
Finished Build : #22 of Job : Stan Pull Request - CppLint Math with status : SUCCESS
Finished Build : #176 of Job : Stan Pull Request - CppLint with status : FAILURE
Build step 'MultiJob Phase' marked build as failure
Setting status of 86bae9184cc29b79094b0601115c47f30d47ea42 to FAILURE with url http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Pull%20Request/462/ and message: 'Build finished.'

Refer to this link for build results (access rights to CI server needed):
http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Pull%20Request/462/

[bob-carpenter]

@dustinvtran Thanks!

@syclik Do you understand this code well enough to review it? I missed the overview and haven't been caught up yet.

[akucukelbir]

@bob-carpenter this is a branch off of feature/bbvb. i can review it and merge it into feature/bbvb. so nothing here is touching develop at the moment.

[bob-carpenter]

@dustinvtran It's only failing in the cpplint style filter, which enforces the Google C++ style. The errors are usually very easy to fix. You can run it locally using make cpplint. See:

https://github.com/stan-dev/stan/wiki/Coding-Style-and-Idioms

[dustinvtran]

Great, thanks. I went ahead and fixed it on both branches.

[akucukelbir]

you're moving too fast for me to react! 👍 thanks dustin.

[stan-buildbot]

Refer to this link for build results (access rights to CI server needed):
http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Pull%20Request/466/