Understanding the AdvancedVI interface documented in the README

[weinstockj]

I'm asking for a bit more clarity on the intended interface for AdvancedVI. For context, I'm a bit confused by the README, which includes the following example:

$$ \begin{aligned} x &\sim \mathrm{LogNormal}\left(\mu_x, \sigma_x^2\right) \\ y &\sim \mathcal{N}\left(\mu_y, \sigma_y^2\right), \end{aligned} $$

using LogDensityProblems

struct NormalLogNormal{MX,SX,MY,SY}
    μ_x::MX
    σ_x::SX
    μ_y::MY
    Σ_y::SY
end

function LogDensityProblems.logdensity(model::NormalLogNormal, θ)
    (; μ_x, σ_x, μ_y, Σ_y) = model
    return logpdf(LogNormal(μ_x, σ_x), θ[1]) + logpdf(MvNormal(μ_y, Σ_y), θ[2:end])
end

function LogDensityProblems.dimension(model::NormalLogNormal)
    return length(model.μ_y) + 1
end

function LogDensityProblems.capabilities(::Type{<:NormalLogNormal})
    return LogDensityProblems.LogDensityOrder{0}()
end

This all makes sense me: we define the intended probability model, and use the LogDensityProblems interface to provide helpful abstractions. I interpret the second parameter of the logdensity function to be where the "data" (in this case, [X, Y] ) would come in. I understand that X is a scalar and Y a vector.

However, later in the sample, I would expect that would simulate X and Y from the true generative model, and then attempt to infer the posterior of the parameters using AdvanedVI.optimize . However, that doesn't occur; the example instantiates the parameters, but does not simulate the 'data' from the true generative model.

Where in this example is the "data" actually fed in?

[Red-Portal]

Hi, thank you for your interest! In that example, there is no notion of data or model. The target distribution (or posterior if you wish) is just the distribution of (x, y). So if you run VI on that problem, you are approximating the distribution of (x, y).

AdvancedVI only expects you provide an unnormalized density (posterior or not), and will try to approximate it.

[weinstockj]

Ah, ok. It seems then the example perhaps had a low density in my personal prior of common models. To offer an unsolicited suggestion - I wonder if the documentation would benefit from the inclusion of a few more examples, perhaps a few with some demonstrations of more common models (e.g., Bayesian linear regression of some kind - perhaps a non-conjugate model).

Thanks for developing AdvancedVI!

[Red-Portal]

Yeah, we'll have to get to that very soon. However, AdvancedVI will still be changing a bit in the near future, so polishing it is unfortunately a low priority for us at the moment. Thanks for the feedback!

[Red-Portal]

The README now describes a more realistic example that should be more intuitive. Thanks for the suggestion!