Likelihood profiles for state-space models

[kellijohnson-NOAA]

First, thank you to @Cole-Monnahan-NOAA bringing this topic up in our Diagnostic Design Google Document, which is only accessible to NOAA employees.

Kelli first suggested that we should encourage FIMS users to run Bayesian models as a diagnostic tool rather than 2d likelihood profiles on MLE models because likelihood profiles do not incorporate the uncertainty of the parameters that are being profiled over and it can lead to overconfidence in things. Additionally, a lot of work goes into fiddling with non-converged runs within the profile, particularly for unstable models.

Cole then suggested that 1-dimensional likelihood profiles are good for testing data conflict, which MCMC cannot easily be used for. But, profiles make much less sense for state-space models. You plot the joint likelihood components but fit to the marginal likelihood. Weird things can happen (non-quadratic shapes) and in general they are much harder to interpret. This deserves a broader conversation and maybe some simulation testing on the properties of profiles when using the Laplace approximation.

So, it would be helpful if people could post links to papers that talk about likelihood profiles with state-space models, ask questions regarding the use of likelihood profiling with state-space models or even non-state-space models, and provide their opinions on the topic. Please tag those that you know will be interested in this topic. Remember to reply in thread if you are replying to someone's question. I look forward to following the conversation 😁.

[Cole-Monnahan-NOAA]

I haven't explored this more or really thought about it much. But, I think the worst case scenario is that you can just fit your model with the LA and then map off the process variance terms and turn off the LA, reverting your model back to a penalized ML model with the variances fixed. Then do the profile like normal. This is equivalent to what we currently do in PML models, but you'd just be using better estimates of the variances.