Question on ModelFinder in IQ-TREE 3: mixture of reversible and non-reversible models

[dongzhang0725]

Dear IQ-TREE development team,

I recently encountered an issue when performing model selection with IQ-TREE 3. Specifically, when I used the following command:

-mset JC69,K80,F81,HKY,TN93,GTR,12.12 -mrate G

I received the error:

ERROR: Add an option '--nonrev-model' when using non-reversible models in ModelFinder.

However, after adding the suggested '--nonrev-model' option, I then encountered another error:

ERROR: The input models cannot contain reversible and non-reversible models

In previous versions of IQ-TREE, this combination of models could run without any problem. Could you please clarify why IQ-TREE 3 no longer supports mixtures of reversible and non-reversible models? Also, is there any recommended workaround or solution to achieve the same functionality?

Thank you very much for your time and support.

Best regards,
Dong Zhang

[thomaskf]

Dear Dong Zhang,
Thank you for your question.
In IQ-TREE 3, mixtures of reversible and non-reversible models are no longer supported during model selection. The reason is that for non-reversible models, the location of the root on the tree affects the likelihood value, whereas for reversible models the likelihood is independent of root location.
In the model selection workflow, the likelihoods of all input models are compared on the same tree, we cannot guarantee that likelihoods from reversible and non-reversible models are directly comparable under this framework. IQ-TREE 3 thus does not allow combining these two classes of models in the same run.
As a workaround, if you would like to consider both types of models, you can perform two separate runs: one with reversible models only, and another with non-reversible models only. You can then compare the best-fitting models from each run.

[dongzhang0725]

Thank you very much for your detailed explanation. I understand the reasoning behind the change in IQ-TREE 3 regarding the separation of reversible and non-reversible models.

However, this modification has introduced considerable inconvenience for several usage scenes. Would it be possible to reconsider supporting the option of including both reversible and non-reversible models together, perhaps with a clear warning message to users about the comparability issues?

Such an approach could preserve flexibility for advanced users who are aware of the limitations, while still maintaining the robustness of the workflow for standard use cases.

Thank you again for your attention and for your continued efforts in developing IQ-TREE.

Best regards,
Dong Zhang

[roblanf]

@dongzhang0725 the issue is that we have realised that the models are not actually comparable. So it really doesn't make sense to allow them in the same analysis. To do so would imply that we think that ranking the models by AIC or BIC (which is what ModelFinder does) is meaningful. That is no longer the case.

For the same reason, we don't allow PMSF models in ModelFinder (in PMSF models, the number of free parameters is undefined).

Our aim is to keep IQ-TREE up to date so that all users can trust the output. So the only reason we'd put reversible and non-reversible models back together in ModelFinder is if/when we can be convinced that the AIC and BIC can be meaningfully used to compare them. (Or if someone comes up with a new metric that can be used, which of course is an active area of research in various places).

To summarise, at the moment, our best understanding is that if you want to choose between reversible or non-reversible models you'll need to come up with comparison methods that do not rely on the AIC or BIC.

[dongzhang0725]

Dear @roblanf

Thank you for the detailed and patient explanation. I now understand why the models are not comparable and why it would be misleading to rank them by AIC or BIC.

I appreciate your commitment to keeping IQ-TREE's output trustworthy and reliable. Your summary clarifies that if a comparison is needed between reversible and non-reversible models, it must be done using methods that do not rely on AIC or BIC. This is very helpful.

Best regards,
Dong Zhang

[evolbeginner]

Dear Dong Zhang, Thank you for your question. In IQ-TREE 3, mixtures of reversible and non-reversible models are no longer supported during model selection. The reason is that for non-reversible models, the location of the root on the tree affects the likelihood value, whereas for reversible models the likelihood is independent of root location. In the model selection workflow, the likelihoods of all input models are compared on the same tree, we cannot guarantee that likelihoods from reversible and non-reversible models are directly comparable under this framework. IQ-TREE 3 thus does not allow combining these two classes of models in the same run. As a workaround, if you would like to consider both types of models, you can perform two separate runs: one with reversible models only, and another with non-reversible models only. You can then compare the best-fitting models from each run.

Thank you for the explanations @thomaskf and @roblanf !

May i bother asking a question? I might misunderstand sth., but as i naively understand, rev and nonrev are nested models? If so, it could mean that they may directly be comparable using even LRT, which i think was used in [Yang, 1994] and perhaps also in others. So could i ask why they are not comparable? In regards to DNA model, NONREV has $4*3-1=11$ subs. rate parameters, plus one root position, while REV has $6+3-1=8$ subs. rate parameters. I just couldn't understand why they are not comparable. I'd be very grateful if you could illustrate a bit more. Thank you in advance!

best,
sishuo

[thomaskf]

Dear Sishuo,

Yes, reversible (rev) and non-reversible (nonrev) models are nested. However, this is not a limitation for AIC or BIC, since these information criteria can be applied even when comparing non-nested models.

The key concern here is that for a nonrev model, the likelihood depends on the location of the root. When comparing a rev model with a nonrev model, AIC or BIC does not consider this additional parameter —- the root position. Because of this, we are not convinced that using AIC or BIC to rank rev versus nonrev models is meaningful, unless there is an evidence in published research showing that such a comparison is sensible.

Thomas

[roblanf]

This discussion has definitely made me think a bit more about this though (thanks @evolbeginner and @dongzhang0725).

One way to think about this is that we could root the rev model in precisely the same place as the non-rev model, and we'd know exactly what the likelihood is (it's the same as the likelihood with any root, of course). This makes me think that the models may be comparable after all.

Thoughts? @bqminh do you have an opinion here? Can anyone think of a way of checking this via simulation? I.e. checking that the model comparisons are meaningful (or not, perhaps they're biased!).

My first impression is that if we simulate sequences (which of course we need to do on a rooted tree) with reversible and non-reversible models, and just ask if the model selection is biased towards one or other type of model. One could also adjust the extent of the non-reversibility - i.e. the distance between the non-reversible model and the nearest reversible equivalent.

We can also ask about the accuracy of tree reconstruction (and root placement, for non-rev models) under the simulating model, the best non-rev model, the best rev model, and the model selected by AIC, AICc, BIC etc.

I think if we were to have this kind of information, we'd be able to decide to what extent we can reliably compare reversible and non-reversible models with existing methods.

I reckon I could still be convinced with a first-principles argument though (in either direction - I am not expert enough to think about this clearly enough on my own).