Tuning during sampling phase

[mikeryanmay]

This topic came up a few times at the SSB meeting and RevBayes OSE workshop last week, so I thought I would bring it up here for a more general conversation. It seems to be common in RevBayes tutorials to use proposal tuning during the entire run, for example from the Simple Diversification Rate Estimation tutorial:

mymcmc.run(generations=50000, tuningInterval=200)

My understanding is that this violates detailed balance, meaning that the samples will not be drawn from the actual posterior distribution (in theory). Detailed balance requires that the probability of being in state i and going to state j is the same as the probability of the reverse transition. If the transition kernel changes for different parts of the posterior distribution then this requirement will not be satisfied (it is not sufficient that the forward and backward transitions under consideration happen at iteration i and i + 1, during which time the proposal kernel is presumably constant).

To demonstrate the potential problem, I set up a toy example where the target distribution is a (equally weighted) mixture of two normals, with mean = 0 and 2 and sds = 0.1 and 0.7. I ran the chain with and without tuning during the sampling phase, and here are the sample CDFs compared to the analytical CDF:

The same issue is apparent but less severe with a lognormal distribution:

The affect depends on the proposal distribution being used. In the above lognormal example I used an mvSlide proposal. Here is the same plot but with a tuned mvScale proposal added (green), which shows that actually the tuned chain seems to be sampling from the correct distribution:

Clearly in this case tuning works, but I think we all agree that getting different posteriors with different proposals is undesirable.

There are certainly cases where the effect is minimal (eg just sampling from a normal distribution), and I did have fiddle around with the model parameters, proposals, and tuning frequency to generate the above plots (though the behavior is very repeatable across runs under the same settings). Maybe in a lot of cases tuning during sampling is innocuous, and I think this is default behavior in BEAST so I understand the motivation. I'm just a bit uncomfortable when the posteriors are unknown and we don't generally understand what the effect of tuning like this is.

What do other people think about this?

(Scripts attached)
bimodal_mixture_untuned.txt
bimodal_mixture_tuned.txt
lognormal_untuned.txt
lognormal_tuned.txt
lognormal_tuned_scale.txt

[jhuelsenbeck]

We should not even allow for tuning during the sampling phase of the MCMC. Perhaps we could have a burn-in or tune phase, and once the tuning parameters have been optimized, we lock them down during the sampling phase. This would also be the case for path sampling, where we could have repeated cycles of burn-tune-sample for different powers. JH

…

On Mon, Jan 19, 2026 at 9:39 PM Michael R May ***@***.***> wrote: This topic came up a few times at the SSB meeting and RevBayes OSE workshop last week, so I thought I would bring it up here for a more general conversation. It seems to be common in RevBayes tutorials to use proposal tuning during the entire run, for example from the Simple Diversification Rate Estimation tutorial <https://revbayes.github.io/tutorials/divrate/simple.html#running-an-mcmc-analysis> : mymcmc.run(generations=50000, tuningInterval=200) My understanding is that this violates detailed balance, meaning that the samples will not be drawn from the actual posterior distribution (in theory). Detailed balance requires that the probability of being in state i and going to state j is the same as the probability of the reverse transition. If the transition kernel changes for different parts of the posterior distribution then this requirement will not be satisfied (it is not sufficient that the forward and backward transitions under consideration happen at iteration i and i + 1, during which time the proposal kernel is presumably constant). To demonstrate the potential problem, I set up a toy example where the target distribution is a (equally weighted) mixture of two normals, with mean = 0 and 2 and sds = 0.1 and 0.7. I ran the chain with and without tuning during the sampling phase, and here are the sample CDFs compared to the analytical CDF: mixture_cdf.jpg (view on web) <https://github.com/user-attachments/assets/27931767-2c93-4707-96fc-813cb023c6c6> The same issue is apparent but less severe with a lognormal distribution: lognormal_cdf.jpg (view on web) <https://github.com/user-attachments/assets/731d5009-2990-4bbe-905f-2705ad7c9329> The affect depends on the proposal distribution being used. In the above lognormal example I used an mvSlide proposal. Here is the same plot but with a tuned mvScale proposal added (green), which shows that actually the tuned chain seems to be sampling from the correct distribution: lognormal_cdf_scale.jpg (view on web) <https://github.com/user-attachments/assets/4d1b9b90-6829-47e8-8a45-22fedecf1177> Clearly in this case tuning works, but I think we all agree that getting different posteriors with different proposals is undesirable. There are certainly cases where the effect is minimal (eg just sampling from a normal distribution), and I did have fiddle around with the model parameters, proposals, and tuning frequency to generate the above plots (though the behavior is very repeatable across runs under the same settings). Maybe in a lot of cases tuning during sampling is innocuous, and I think this is default behavior in BEAST so I understand the motivation. I'm just a bit uncomfortable when the posteriors are unknown and we don't generally understand what the effect of tuning like this is. What do other people think about this? (Scripts attached) bimodal_mixture_untuned.txt <https://github.com/user-attachments/files/24723382/bimodal_mixture_untuned.txt> bimodal_mixture_tuned.txt <https://github.com/user-attachments/files/24723383/bimodal_mixture_tuned.txt> lognormal_untuned.txt <https://github.com/user-attachments/files/24723384/lognormal_untuned.txt> lognormal_tuned.txt <https://github.com/user-attachments/files/24723385/lognormal_tuned.txt> lognormal_tuned_scale.txt <https://github.com/user-attachments/files/24723386/lognormal_tuned_scale.txt> — Reply to this email directly, view it on GitHub <#934>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB573Q44523MWQUMW3ZZERL4HU6JDAVCNFSM6AAAAACSGMGPRGVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZZGM3DIMRUGM> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- John Huelsenbeck Department of Integrative Biology University of California, Berkeley Berkeley, CA 94720 (510) 502-5887 ***@***.***

[bredelings]

This is a nice example that does a great job illustrating the problem. I think we need to stop the current practice of unlimited tuning immediately.

I guess the question is how we do that. The website contains 145 examples of tuning while sampling. So we need to fix these examples on the website.

Another question is what interface for tuning we want. Normally I would think the question is "how many generations until tuning stops". RevBayes seems to have kind of a different interface, where you explicitly label some iterations as burnin in the script itself. However, sometimes you don't know how much burnin you need until you run the chain, so some of the iterations from mcmc.run( ) might still be dropped as burnin.

My impression is that tuning generally takes a shorter time than burnin. So it could still make sense to only allow tuning during mcmc.burnin( ) and not during mcmc.run( ).

[mikeryanmay]

I think the ship has already sailed on mcmc.burnin( ), but it is often difficult to explain to people that they still might have to throw out samples that were generated by mcmc.run() after they did an mcmc.burnin(). Probably a better name for the current mcmc.burnin( ) would be mcmc.warmup( ) or mcmc.tune( ).

[davidcerny]

@hoehna has been consistently calling it pre-burnin. The idea is that "pre-burnin" is the period during which you don't draw samples at all, whereas "burnin" refers to those samples that the user decides to discard after the fact.

[bredelings]

One thing we could do is to continue accepting the tuning argument to mcmc.run( ) and friends, but (i) make it have no effect and (ii) issue a deprecation message that says "This is wrong, do instead, this will become an error in the future". Then in a future release we could remove the tuningInterval argument from mcmc.run( ) entirely.

[mikeryanmay]

This would be my preference. Adding an additional argument like tuneUntilGeneration may just increase confusion among users.

[bredelings]

Another option would be to add a tuneUntilGeneration argument and default it to zero.

We could make this issue a warning if the tuning is greater than (say) 10% of the total number of iterations.

I don't know. Thoughts?

[mikeryanmay]

Is the thought here that this would be an argument for mcmc.run()? Would there be samples written to file during this phase?

[ms609]

a warning if the tuning is greater than 10% of the total number of iterations

This assumes that the total number of iterations is known in advance – not clear how this would play in an analysis that uses stopping rules to determine the length of the run. Would stopping rules need to exclude the 'tuned' generations from their calculations?

[davidcerny]

Another thing to consider here would be adaptive MCMC moves – my understanding is that these deliberately do what's being criticized here, i.e., change the proposal distribution on the fly while collecting samples.

[bredelings]

Can you name a specific example?

[davidcerny]

The BEAST 2 folks have several examples here:

https://beast.community/adaptive_mcmc

We are introducing an adaptive move (mvAdaptiveRJSwitch) on the hopefully soon-to-be-merged dev_stairwayplot branch:

src/core/moves/proposal/mixture/AdaptiveReversibleJumpProposal.cpp‎
src/revlanguage/moves/mixture/Move_AdaptiveReversibleJumpSwitch.cpp‎
‎help/md/mvAdaptiveRJSwitch.md‎

[mikeryanmay]

RevBayes also has mvAVMVN, which I believe is more-or-less a reimplementation of Baele et al. 2017. It has an argument maxUpdates that ensures the covariance matrix becomes fixed after a certain point.

[bredelings]

I guess I mean papers from statistics people that prove that you get the right equilibrium. The BEAST folks are referencing Haario (1999), which doesn't provide a proof that they get the right equilibrium distribution. Worryingly, their approach is non-Markovian. So technically, what they are doing is not even MCMC.

Haario et al later prove that it has the right equilibrium conditional on certain requirements being met. For example, if the size of the adaptation gets smaller and smaller, so that eventually it mostly stops adapting. Note that in RevBayes, this is usually not the case -- we just let adaptation continue forever without getting smaller.

Merging a move with a specific adaptation strategy could be OK if it comes with a proof that this specific strategy doesn't change the equilibrium distribution. But I imagine such a strategy would be contained in the move itself.

[davidcerny]

This looks relevant: https://link.springer.com/article/10.1007/s11222-008-9110-y

[bredelings]

Yep.

[bjoelle]

So the BEAST folks reference Baele 2017, which compares results from a run with and without adaptive moves. The results shown in the paper are only about performance, but I assume they would have also checked that they got the same estimates (or at least that we could recheck that fairly easily).
For non-adaptive moves, since tuning is on by default in BEAST and BEAST2 I would assume that we would have seen it earlier if it was an issue, no ? If nothing else the validation of new models should fail since the distributions would not be correct.

[bredelings]

So the BEAST folks reference Baele 2017, which compares results from a run with and without adaptive moves. The results shown in the paper are only about performance, but I assume they would have also checked that they got the same estimates (or at least that we could recheck that fairly easily).

So I looked at the Beale 2017 paper and it relies on proofs in other papers to justify the adaptive strategy:

Note that the chosen probability for the acceptance resembles the familiar acceptance probability of the Metropolis algorithm. This choice for the acceptance probability is not based on symmetry/reversibility conditions as these cannot be satisfied, given that the corresponding stochastic chain is no longer Markovian [Haario et al., 2001]. It is known that adaptive MCMC algorithms will not always preserve stationarity of π [Roberts and Rosenthal, 2009]. However, having proven the ergodicity of adaptive MCMC under certain conditions [Roberts and Rosenthal, 2007], the authors have also shown that the AM algorithm above will indeed converge to π and satisfy the Weak Law of Large Numbers, even though it is not Markovian.

So, in this case it is fine to continue tuning. Basically, they estimate the covariance from the entire chain history instead of just the recent past, so the effect of new samples goes down over time. It also prevents narrowing the proposal distribution if you happen to be currently exploring a narrower mode in a bimodal distribution, like in Mike's example above.

However, the tuning for mvSlide does not decrease the adaptation over time, so it is not fine to continue tuning indefinitely.

[bredelings]

I'm curious about the BEAST papers that do adaptation and then cite Baele 2017. Baele 2017 implements a specific adaptation strategy for MVN that is proven to work, but it doesn't say "adapation is fine in general". I see that Douglas et al 2021 "Adaptive dating and fast proposals" implements tuning for operator weights, but makes no argument that tuning is OK. I can't quite tell if the proposal weights are fixed in the "final" phase, but it looks like they are not. So an argument is needed. I note that unlike the Baele 2017 paper, Suchard is not an author, which could explain the mistake, if there is a mistake.

[bjoelle]

ok I checked the BEAST2 moves and I don't see any explicit mechanism to stop tuning after a while, but it does take into account the entire history of the move (# of accepted + # of rejected) and includes 1/count in the size of the correction, so I would expect the tuning to reduce to close to 0 as the chain progresses
the adaptive moves in BEAST2 are a more recent implementation but from memory of discussions with Remco they are also supposed to settle progressively

[mikeryanmay]

ok I checked the BEAST2 moves and I don't see any explicit mechanism to stop tuning after a while, but it does take into account the entire history of the move (# of accepted + # of rejected) and includes 1/count in the size of the correction, so I would expect the tuning to reduce to close to 0 as the chain progresses

This is reassuring, certainly better than continuously tuning as is possible for the mast majority of existing revbayes moves. My own preference would be that the adjustment is eventually stopped, similar to mvAVMVN. Obviously that would require a great deal of re-engineering.

Edit: to be less implicit, given the way revbayes currently implements tuning, allowing it to occur during mcmc.run() is not justifiable. In lieu or re-engineering the moves to have tuning parameters that become fixed over time (and behave appropriately for sequences of chains where re-tuning may be appropriate, e.g., stepping stones), I think tuningInterval for mcmc.run() should be deprecated.

[bredelings]

I was looking for whether there was anything more sensible that we could do than the current tuning formula. One simple, common approach is Robbins-Monro tuning, which does decrease adaptation over time. It seems this is what BEAST X / BEAST 2 are using from back before they split.

https://www.beast2.org/2026/02/01/automatic-operator-tuning.html

One question is whether the current formula could oscillate around a good value.