Investigate the use of limited information posterior/likelihood for inference

[ben18785]

If we have a normal sampling model of the form,

y(t) ~ N(f(theta, t), sigma),

then, if we assume a uniform prior for sigma ~ U(0, upper), then if the sum of squared errors (SSE) = sum_i=1^n (y(t) - f(theta, t))^2, then the analytic posterior is given by,

1/2 SSE^(1/2 - n/4) Gamma(1/4 (-2 + n), SSE/upper^2) p(theta),

where Gamma(a,b) is the incomplete gamma function (scipy.special.gammaincc(a,b) in Python), and p(theta) is the prior over the model parameters.

This means that these types of normal sampling problems can be written as an inference/optimisation problem with one fewer dimensions. Whether or not this is useful generally, I don't know. It depends how correlated sigma is with elements of theta, I suppose?

It does, however, make it feasible to use deterministic numerical integration for the logistic equation to numerically determine the marginal likelihood for small enough datasets, which is useful.

Note to self: the above logic still holds for a few other select p(sigma), albeit with different algebraic results. For example, p(sigma) ~ (1/sigma^2) exp(-1/sigma^2) yields an analytic form for the posterior. These p(sigma) could be used for more informative priors.