Cross-model proposals are problematic

[pgrinaway]

I've noticed that certain cross-model proposals (specifically OBC1 and OBC2 to HCT) are very bad, and almost always result in rejection. I figured I'd post this to have some discussion of possible remedies.

[jchodera]

Can you compute an empirical transition matrix? If you write the model every iteration, you can compute P(model at t+1 is j | model at t was i) = T_{ij}. There are many ways to increase the acceptance probability for RJMC. The simplest for GB models would likely be to (1) either use a parameterization of variables that makes the GB functions more similar to each other for the same value of parameters, or (2) use proposal distributions to sample new parameters for the new model that attempt to make the functions more similar for the new proposal. If we're just using an index in pymc instead of actual RJMC proposals, option (1) may be easiest, but would likely also necessitate incorporating a Jacobian in the prior to ensure that the posterior is independent of the variable parameterization we choose. That effect may not be strong, so we could potentially skip this. Will look at the functional forms in a moment, but the essence of this is to plot the value of one effective born radius function and the other born radius function across their domains and try to find a parameterization for each function that makes the effective born radius more similar for the same base parameter theta.

[jchodera]

HCT is also a very old model, and might simply be a poor quality model. Do we have a way to distinguish between poor mixing and highly disfavored models?

[pgrinaway]

Can you compute an empirical transition matrix? If you write the model
every iteration, you can compute P(model at t+1 is j | model at t was i) =
T_{ij}.

Sounds good!

(1) either use a parameterization
of variables that makes the GB functions more similar to each other for the
same value of parameters

I agree that this could be the simplest way. Given that the functional forms are not that complex, I don't think that a deterministic transformation would have a particularly hairy Jacobian, right?

(2) use proposal distributions to sample new
parameters for the new model that attempt to make the functions more
similar for the new proposal.

I'll look into this as well.

HCT is also a very old model, and might simply be a poor quality model. Do we have a way to distinguish between poor mixing and highly disfavored models?

Good point! I was saying that the moves are difficult because it seemed that a proposed transition to HCT resulted in NaNs, but I agree that it is important to distinguish between the two. I ran a few chains last night, and I'll take a look at them today to see how it went.

[jchodera]

I wouldn't be too worried about the NaN issue unless it is happening often, in which case it might be a numerical robustness issue that we should raise with OpenMM.