Add mean function support for spatiotemporal GPs

[sethaxen]

This PR addresses two issues:

• Though children of SpatioTemporalBase take a mean function as argument, they don't seem to use the function at all.
• The mean functions currently implemented all support only a single temporal dimension and no spatial dimensions.

This PR adds support for using separable spatial and temporal mean functions within the current children of SpatioTemporalBase. It does this by adding an abstract class SpatioTemporalMeanFunction and a subclass SeparableMeanFunction, which is the sum of a spatial mean function (should be a GPflow mean function) and a temporal mean function (should be a markovflow mean function).

I'm not certain whether the separability is a requirement or not, but since ConditionalPosterior needs a pure temporal mean function, it seemed much simpler to support these than to support all possible spatiotemporal mean functions. I'm also not convinced I have overloaded all methods of SpatioTemporalBase necessary for all functionality to correctly use the mean function.

@vincentadam87 does this look like a suitable approach? If so I will

• add a unit test for the new mean function
• add a notebook demonstrating use of a spatial mean function, which will also be an integration test
• add an integration test using the mean function

[vincentadam87]

I think you can't really distribute the temporal and spatial means across the prediction the way you do (see detailed comment)
So there is not a real need to have the Separable vs non separable mean abstraction.

yes for test! (not sure what scenario makes sense though, we do not have tests for spatio temporal at all actually, bad. any test welcomed)

notebook: we have one, maybe just add a mean there in the existing demo?

[sethaxen]

@vincentadam87 I have implemented the mean function approach you suggested (adding the mean function at the end of space_time_predict). With this mean function specified, the tests implemented in #23 now pass for SpatioTemporalSparseVariational, but for SpatioTemporalSparseCVI both the ELBO calculation and the mean prediction disagree with GPR (EDIT: varying the learning rate or the number of times update_sites is called has no effect). Are there other places in SpatioTemporalSparseCVI where the mean function needs to be taken into account?

[vincentadam87]

my hunch is that you need to substract the mean function somewhere along the way in the site_update calculation.

I would need to do the math properly but it is possibly in the input to the 'gradient_transformation_mean_var_to_expectation'

[sethaxen]

my hunch is that you need to substract the mean function somewhere along the way in the site_update calculation.

I would need to do the math properly but it is possibly in the input to the 'gradient_transformation_mean_var_to_expectation'

That did the trick! Now the tests pass. This approach ends up computing the mean function once more than is necessary, but I don't believe this is the computational bottleneck, and doing it differently would require more code restructuring.

[vincentadam87]

Looks perfect to me!