Gaussian process functions and classes

[fonnesbeck]

Replaces gp_classes branch. Will eventually include a GP class, mean functions, and a function for drawing realizations from the GP after sampling.

[fonnesbeck]

Getting close, but the GP class returns biased estimates.

[fonnesbeck]

Have a peek at the interface. I have updated the GP-introduction notebook in this PR to use it. Predictive sampling now happens outside of MCMC sampling, using sample_gp, which uses the same API as sample_ppc. This seems to make sense to me, but would like to hear what others think.

/cc @bwengals

[twiecki]

API looks great! Would the new ESS work well here?

[bwengals]

This is awesome! Yeah, this looks good to me. Also, looks like NUTS and ADVI did well! I had trouble getting those to work. That is very cool.

[fonnesbeck]

@bwengals NUTS and ADVI did not work well on the second example -- ADVI kept getting nans.

[bwengals]

Oh, huh. I wonder why. Well I like the new GP class.

[fonnesbeck]

Using two-stage integration seemed to help quite a lot with the second example.

[fonnesbeck]

Need to write some tests and docs, and we should be good.

[twiecki]

can you pass integrator='two-stage' to sample() to still get ADVI init? We might need to improve the kwarg forwarding.

[fonnesbeck]

ADVI init kept failing with NaN errors, so I turned it off.

[fonnesbeck]

Unusual test failure only on 2.7. Is it a different version of Theano there?

[fonnesbeck]

This appears to pass now. I will add docs in a separate PR.

[twiecki]

This is great

[AustinRochford]

Yes, this is really exciting! I have an old talk on Bayesian optimization that I'm excited to port to pymc3

[fonnesbeck]

I'd be interested in seeing that!