Add base_conditional_with_lm function

[nfergu]

PR content:

• Title
• Description
• Minimal working example

Title

Allow Lm to be passed into utility functions for computation of conditionals.

Description

This PR adds a new utility method: base_conditional_with_lm in conditionals/util.py which accepts Lm instead of Kmm. This enables the performance of, for example, prediction to be improved by passing a pre-computed value for Lm into this function, instead of calculating it each time.

This is a hopefully (improved) version of #1514, which uses two functions instead of trying to put both Kmm and Lm into a single function.

Minimal working example

Here is a rough example of how I would use the new base_conditional_with_lm function in my model. This improves prediction time by about 10x.

class MySVGP(SVGP):

    def __init__(self, kernel, likelihood, inducing_variable, *, mean_function=None,
                 num_latent_gps: int = 1, q_diag: bool = False, q_mu=None, q_sqrt=None,
                 whiten: bool = True, num_data=None):
        super().__init__(kernel, likelihood, inducing_variable, mean_function=mean_function,
                         num_latent_gps=num_latent_gps, q_diag=q_diag, q_mu=q_mu, q_sqrt=q_sqrt,
                         whiten=whiten, num_data=num_data)
        self._Kmm_for_prediction = None  # [M, M]
        self._Lm_for_prediction = None  # [M, M]

    def train(self):
        # Do training here...
        self._Kmm_for_prediction = covariances.Kuu(self.inducing_variable,
                                                   self.kernel, jitter=default_jitter())
        self._Lm_for_prediction = tf.linalg.cholesky(self._Kmm_for_prediction)

    def predict_f(self, Xnew: InputData, full_cov=False, full_output_cov=False) -> MeanAndVariance:
        if self._Kmm_for_prediction is None or self._Lm_for_prediction is None:
            raise ValueError('Kmm or LM is not set. Use the train method to train the model.')
        mu, var = self.compute_conditional(Xnew, full_cov, full_output_cov)
        return mu + self.mean_function(Xnew), var

    def compute_conditional(self, Xnew, full_cov, full_output_cov):
        Kmn = Kuf(self.inducing_variable, self.kernel, Xnew)  # [M, N]
        Knn = self.kernel(Xnew, full_cov=full_cov)
        fmean, fvar = base_conditional_with_lm(
            Kmn, self._Lm_for_prediction, Knn, self.q_mu, full_cov=full_cov,
            q_sqrt=self.q_sqrt, white=self.whiten
        )  # [N, R],  [R, N, N] or [N, R]
        return fmean, expand_independent_outputs(fvar, full_cov, full_output_cov)

[nfergu]

@st-- Based on your feedback on #1514 I have created a new PR to try and solve this.

I thought that instead of the approach you suggested of having a Kmm_is_cholesky flag it might be cleaner to have two separate functions (one of which calls the other). What do you think of this approach?

If you think this is a good approach then I can add some tests and hopefully we can get this merged.

[codecov]

Codecov Report

Merging #1528 into develop will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff            @@
##           develop    #1528   +/-   ##
========================================
  Coverage    95.30%   95.31%           
========================================
  Files           85       85           
  Lines         3815     3817    +2     
========================================
+ Hits          3636     3638    +2     
  Misses         179      179           

Impacted Files	Coverage Δ
gpflow/conditionals/util.py	76.36% <100.00%> (+0.21%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d995194...b38f351. Read the comment docs.

[nfergu]

@st-- Any thoughts on this approach? We'd like to incorporate this GPflow change into our models fairly soon, if we can.

[st--]

I'd be happy to merge this but unfortunately codecov-project seems to be having issues... :S

[nfergu]

@st-- Thanks. Do you know if there's any way to get codecov to try again? I could try and push an insignificant change this to PR to try and trigger it, but there might be a better way?