structured mass matrices

[bob-carpenter]

It'd be nice to have structured mass matrices so that we could have a mass matrix where there are structured zeros. The use case is when we think there are correlations in the posterior among some of the parameters that should go into the mass matrix. The particular use case motivating this is a model with a dozen or so parameters and then a whole lot of item-level parameters (what some people might call "random effects" if they wanted to annoy Andrew).

• linear algebra for structured mass matrices
• command-line options to specify structure
• integration with samplers

[bgoodri]

Is it easier / better to go with structured mass matrices or batched updates of parameters that have a batch-specific mass matrix? In your example, you could alternate between updating the dozen parameters with a dense mass matrix and the item-level parameters with a diagonal mass matrix. I guess this would require us to implement parameter tagging.

With a single mass matrix, it seems like you would need to use an Eigen::SparseMatrix or some sort of block matrix scheme. But there currently is no eigenvalues algorithm for an Eigen::SparseMatrix, so we couldn't integrate this with RMHMC, unless you did fill-in, which would be expensive.

[bob-carpenter]

We should probably mention this in the manual w.r.t. user modeling,
where the user can easily break things apart if the matrix block
structure is known, for instance.

Banded structure's even trickier --- my understanding is that
it's not uncommon in something like Gaussian processes due to
hard thresholding on distances of effects. Is there a way to
decompose these in our modeling language?

• Bob

On 12/10/13, 11:43 AM, bgoodri wrote:

Is it easier / better to go with structured mass matrices or batched updates of parameters that have a batch-specific
mass matrix? In your example, you could alternate between updating the dozen parameters with a dense mass matrix and the
item-level parameters with a diagonal mass matrix. I guess this would require us to implement parameter tagging.

With a single mass matrix, it seems like you would need to use an Eigen::SparseMatrix or some sort of block matrix
scheme. But there currently is no eigenvalues algorithm for an Eigen::SparseMatrix, so we couldn't integrate this with
RMHMC, unless you did fill-in, which would be expensive.

—
Reply to this email directly or view it on GitHub #426 (comment).

[bgoodri]

I'm not sure if we have anything useful to say in the manual that Stan
supports at this point. Basically, if someone has a structured matrix as a
parameter of their model, then they have to construct it the brute force
way for the most part. I think it is not even possible to assign to a block
of a matrix in the Stan language.

In any event, my main point was just to brainstorm whether batch updates
would be an alternative to a single, giant, structured mass matrix in the
momentum part of the model.

On Tue, Dec 10, 2013 at 3:31 PM, Bob Carpenter notifications@github.comwrote:

We should probably mention this in the manual w.r.t. user modeling,
where the user can easily break things apart if the matrix block
structure is known, for instance.

Banded structure's even trickier --- my understanding is that
it's not uncommon in something like Gaussian processes due to
hard thresholding on distances of effects. Is there a way to
decompose these in our modeling language?

• Bob

On 12/10/13, 11:43 AM, bgoodri wrote:

Is it easier / better to go with structured mass matrices or batched
updates of parameters that have a batch-specific
mass matrix? In your example, you could alternate between updating the
dozen parameters with a dense mass matrix and the
item-level parameters with a diagonal mass matrix. I guess this would
require us to implement parameter tagging.

With a single mass matrix, it seems like you would need to use an
Eigen::SparseMatrix or some sort of block matrix
scheme. But there currently is no eigenvalues algorithm for an
Eigen::SparseMatrix, so we couldn't integrate this with
RMHMC, unless you did fill-in, which would be expensive.

—
Reply to this email directly or view it on GitHub <
https://github.com/stan-dev/stan/issues/426#issuecomment-30244232>.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/426#issuecomment-30264913
.