model comparison

[PTWaade]

Need to save loglikelihoods - should be optional

[zeyus]

This is done with @addlogprob! right? It would also be possible to return an array (then use generated_quantities())

I was thinking to avoid an if statement in the model code it could be wrapped in a function that is created at runtime (before model execution) and either does nothing, or updates the log likelihood if it is enabled?

[PTWaade]

Hmm no there are ways ot do it.

Here :
https://discourse.julialang.org/t/waic-or-loo-etc-in-turing/30873

And using ArviZ.jl is nice:
https://julia.arviz.org/ArviZ/stable/

It's a wrapper for the python ArviZ:
https://www.arviz.org/en/latest/

Basically, it will do most of the things for us for free. All we need to do is to give it the right outputs - you can see what is needed here:
https://julia.arviz.org/ArviZ/stable/quickstart/

I was thinking that perhaps fit_model could output a struct with all that information, and also the priors used and the agent etc.
Then plots and so on can just dispatch on that, and there can be a to_arviz function or somethin.

Might also be simpler ways of doing it.

[PTWaade]

This wpuld be the best model comparison metric:
https://python.arviz.org/en/latest/api/generated/arviz.psislw.html

But there is also WAIC