Gaussian Process support

[brendanhasz]

Currently I can't think of a good way to fit nonparametric models (like Gaussian processes) into the ProbFlow framework. For example, the probflow.model.predict interface would have to change, and all the methods which depend on it, since you'd need both the x and y training data as well as the x test data.

That said, you can still use ProbFlow to fit GP kernel and/or latent parameters. For example, fitting the kernel parameters would look something like:

import tensorflow_probability as tfp
import probflow as pf

class GP(pf.Model):

    def __init__(self):
        self.amplitude = pf.ScaleParameter()
        self.length_scale = pf.ScaleParameter()
        self.var = pf.ScaleParameter()

    def __call__(self, x):
        kernel = tfp.positive_semidefinite_kernels.ExponentiatedQuadratic(
            amplitude=self.amplitude(),
            length_scale=self.length_scale())
        return tfp.distributions.GaussianProcess(kernel, x, 
            observation_noise_variance=self.var())

Which can be fit just fine, but then runs into issues if you were to call predict on the model, since the model doesn't store the training data.

For now I've put further support for GPs on the "out-of-scope" list, but if anyone has ideas for how to make GPs work with the rest of the framework (say, with probflow.model.predict), and want to see that be part of the package, I'm definitely open to discussion!