[BUG] -madd not working in MFP and MF with IQtree3

[dorohuchon]

Hi
I would like to inform that iqtree3 is not testing Cxx models when using -madd.
(which was the option recommended by IQ-GPT)

It does it when using -mset, but then it tests a lot of strange options

For example, using the following command:
iqtree3 -s /IQtree_test.phy -m MFP -mset C10 -T 5 --prefix /dorotheeh/huchond/IQtree_test/IQTREE_MFP_test --mem 100G

I get the following in the .log file

No. Model -LnL df AIC AICc BIC
Model C10 is alias for POISSON+G+FMIX{C10pi1:1:0.1191344178,C10pi2:1:0.0874372456,C10pi3:1:0.1037105070,C10pi4:1:0.0922584809,C10pi5:1:0.1070492801,C10pi6:1:0.1329945166,C10pi7:1:0.0538028458,C10pi8:1:0.0691986212,C10pi9:1:0.1319937434,C10pi10:1:0.1024203429}
1 C10+G4 8439.025 10 16898.050 16898.253 16948.017
Model C10+I is alias for POISSON+G+FMIX{C10pi1:1:0.1191344178,C10pi2:1:0.0874372456,C10pi3:1:0.1037105070,C10pi4:1:0.0922584809,C10pi5:1:0.1070492801,C10pi6:1:0.1329945166,C10pi7:1:0.0538028458,C10pi8:1:0.0691986212,C10pi9:1:0.1319937434,C10pi10:1:0.1024203429}+I
2 C10+I+G4 8439.795 11 16901.590 16901.834 16956.554
Model C10+G is alias for POISSON+G+FMIX{C10pi1:1:0.1191344178,C10pi2:1:0.0874372456,C10pi3:1:0.1037105070,C10pi4:1:0.0922584809,C10pi5:1:0.1070492801,C10pi6:1:0.1329945166,C10pi7:1:0.0538028458,C10pi8:1:0.0691986212,C10pi9:1:0.1319937434,C10pi10:1:0.1024203429}+G
3 C10+G4 8439.025 10 16898.050 16898.253 16948.017
Model C10+I+G is alias for POISSON+G+FMIX{C10pi1:1:0.1191344178,C10pi2:1:0.0874372456,C10pi3:1:0.1037105070,C10pi4:1:0.0922584809,C10pi5:1:0.1070492801,C10pi6:1:0.1329945166,C10pi7:1:0.0538028458,C10pi8:1:0.0691986212,C10pi9:1:0.1319937434,C10pi10:1:0.1024203429}+I+G
4 C10+I+G4 8439.795 11 16901.590 16901.834 16956.554
Model C10+R2 is alias for POISSON+G+FMIX{C10pi1:1:0.1191344178,C10pi2:1:0.0874372456,C10pi3:1:0.1037105070,C10pi4:1:0.0922584809,C10pi5:1:0.1070492801,C10pi6:1:0.1329945166,C10pi7:1:0.0538028458,C10pi8:1:0.0691986212,C10pi9:1:0.1319937434,C10pi10:1:0.1024203429}+R2
WARNING: Both Gamma and FreeRate models were specified, continue with FreeRate model
5 C10+R2 8437.028 11 16896.055 16896.300 16951.019
Model C10+R3 is alias for POISSON+G+FMIX{C10pi1:1:0.1191344178,C10pi2:1:0.0874372456,C10pi3:1:0.1037105070,C10pi4:1:0.0922584809,C10pi5:1:0.1070492801,C10pi6:1:0.1329945166,C10pi7:1:0.0538028458,C10pi8:1:0.0691986212,C10pi9:1:0.1319937434,C10pi10:1:0.1024203429}+R3
WARNING: Both Gamma and FreeRate models were specified, continue with FreeRate model
6 C10+R3 8437.105 13 16900.210 16900.548 16965.167
Model C10+I+R2 is alias for POISSON+G+FMIX{C10pi1:1:0.1191344178,C10pi2:1:0.0874372456,C10pi3:1:0.1037105070,C10pi4:1:0.0922584809,C10pi5:1:0.1070492801,C10pi6:1:0.1329945166,C10pi7:1:0.0538028458,C10pi8:1:0.0691986212,C10pi9:1:0.1319937434,C10pi10:1:0.1024203429}+I+R2
WARNING: Both Gamma and FreeRate models were specified, continue with FreeRate model
14 C10+I+R2 8438.066 12 16900.132 16900.420 16960.092
Model C10+I+R3 is alias for POISSON+G+FMIX{C10pi1:1:0.1191344178,C10pi2:1:0.0874372456,C10pi3:1:0.1037105070,C10pi4:1:0.0922584809,C10pi5:1:0.1070492801,C10pi6:1:0.1329945166,C10pi7:1:0.0538028458,C10pi8:1:0.0691986212,C10pi9:1:0.1319937434,C10pi10:1:0.1024203429}+I+R3
WARNING: Both Gamma and FreeRate models were specified, continue with FreeRate model
15 C10+I+R3 8438.092 14 16904.185 16904.574 16974.138
Model C10+F is alias for POISSON+G+FMIX{C10pi1:1:0.1191344178,C10pi2:1:0.0874372456,C10pi3:1:0.1037105070,C10pi4:1:0.0922584809,C10pi5:1:0.1070492801,C10pi6:1:0.1329945166,C10pi7:1:0.0538028458,C10pi8:1:0.0691986212,C10pi9:1:0.1319937434,C10pi10:1:0.1024203429}+F
23 C10+G4 8439.025 10 16898.050 16898.253 16948.017
Model C10+F+I is alias for POISSON+G+FMIX{C10pi1:1:0.1191344178,C10pi2:1:0.0874372456,C10pi3:1:0.1037105070,C10pi4:1:0.0922584809,C10pi5:1:0.1070492801,C10pi6:1:0.1329945166,C10pi7:1:0.0538028458,C10pi8:1:0.0691986212,C10pi9:1:0.1319937434,C10pi10:1:0.1024203429}+F+I
24 C10+I+G4 8439.795 11 16901.590 16901.834 16956.554
Model C10+F+G is alias for POISSON+G+FMIX{C10pi1:1:0.1191344178,C10pi2:1:0.0874372456,C10pi3:1:0.1037105070,C10pi4:1:0.0922584809,C10pi5:1:0.1070492801,C10pi6:1:0.1329945166,C10pi7:1:0.0538028458,C10pi8:1:0.0691986212,C10pi9:1:0.1319937434,C10pi10:1:0.1024203429}+F+G
25 C10+G4 8439.025 10 16898.050 16898.253 16948.017
Model C10+F+I+G is alias for POISSON+G+FMIX{C10pi1:1:0.1191344178,C10pi2:1:0.0874372456,C10pi3:1:0.1037105070,C10pi4:1:0.0922584809,C10pi5:1:0.1070492801,C10pi6:1:0.1329945166,C10pi7:1:0.0538028458,C10pi8:1:0.0691986212,C10pi9:1:0.1319937434,C10pi10:1:0.1024203429}+F+I+G
26 C10+I+G4 8439.795 11 16901.590 16901.834 16956.554

I am running IQ-TREE 3.0.1 on a SLURM cluster.

Best,

Dorothee

[StefanFlaumberg]

Hi @dorohuchon,

Regarding the -madd option:
The option didn't worked well in the v3.0.1, but has recently been fixed due to the issue report #18.
If you download and compile the master branch, the following command will run fine and test LG+G4, LG+F+G4 and C10 models:
iqtree3 -seed 123 -nt 1 -s someprot.faa -m "MF" -mset "LG" -mrate "+G" -madd "C10" -pre testmadd
Otherwise, just wait until the devs drop the new version, this fix will be included there.

Regarding your results:
According to your command, you are testing the C10 model (the -mset option), with or without the +F state frequency component (testing them both is the default behaviour, when the -mfreq option is not used), and with +E or +I or +G4 or +I+G4 or +Rx or +I+Rx rates (testing them all is the default behaviour, when the -mrate option is not used). This is all in accordance with the IQ-Tree manual.
Then, the C10 model is itself a short name for the POISSON+FMIX{...}+G4 model. ModelFinder uses this long name and adds other specifiers (like +F or +I+R3), then it tries to make sense of this combined name and test the relevant model. Sometimes it results in the same model being tested several times, e.g. the fisrt rows of your output:

1. C10 -> POISSON+FMIX{...}+G4
2. C10+I -> POISSON+FMIX{...}+G4+I
3. C10+G4 -> POISSON+FMIX{...}+G4+G4 -> POISSON+FMIX{...}+G4 (the same as 1)
4. C10+I+G4 -> POISSON+FMIX{...}+G4+I+G4 -> POISSON+FMIX{...}+G4+I (the same as 2)

Note that the free rate always have priority over the gamma rate, i.e. ...+G4+Rx -> ...+Rx.

What is, however, wrong here is that combinations with +F (like C10+F) are treated as if there were no +F in them (compare models 1 and 23 in the ModelFinder output).

Regarding the best way to include the +Cxx mixture state frequencies for testing:
I think neither of the above provides what one usually wants from testing the +Cxx models with ModelFinder. One most likely would like testing the Cxx state frequency mixture combined with different exchange rate matrices (LG, Q.pfam, etc.) and with different types of rate heterogeneities (+I, +I+G, +R, etc.). To this end, we can provide the +Cxx with the -mset option alongside other state frequency types, for example:
iqtree3 -seed 123 -nt 3 -s realprot.phy -m "MF" -mset "LG,WAG" -mfreq "+FU,+F,+C10,+C10+F" -mrate "+G,+I+G" -pre testfreq

Best,
Stefan

[StefanFlaumberg]

Dear @bqminh and @roblanf,

Two relevant points from this issue that might be worth paying attention to:

1. Bug in the ModelFinder that is not fixed in the master branch:
   When specifying -mset "C10", the model combinations with +F (like C10+F) are treated as if there were no +F in them (compare models 1 and 23 in the ModelFinder output above).
2. The IQ-Tree Manual really says nothing about including the +Cxx models for a ModelTest run! I believe examples with -madd "C10,C10+F" and with -mfreq "+F,+C10,+C10+F" should definitely appear there for user convenience.

A more speculative (maybe more suitable for the Discussion section) point that I've wanted to raise for a long time:

The Cxx model really meaning the POISSON+FMIX{...}+G4 model is an utterly counterintuitive thing! I can literally find a bunch of questions and misinterpretations in IQ-Tree github and (unfortunately now unavailable) google-groups regarding this implicit translation.
Among the most confusing consequences:
For a ModelFinder run one can provide a Cxx model with a number of options: 1. -mset "C10", 2. -mfreq "+C10", 3. -madd "C10", and 4. (most surprisingly!) -mrate "+C10". And all these variants run different analyses! As discussed above, with the -mfreq "+C10" variant being the most useful.
How should one specify a somewhat exotic model like LG+C40+G10? With -m "LG+C40+G10"? No, this will run the usual LG+C40+G4 model with 4 gamma categories. The right command is, quite unobviously, -m "LG+G10+C40" (swapped Cxx and Gy). And what about using no rate heterogeneity at all?

My question is thus:
Shouldn't the Cxx labels be reassigned to denote only the +FMIX{...} frequencies with particular predefined state vectors of the original Cxx models? So that something like -m "C10" would not make sense and something like -m "LG+C10" would mean literally LG+FMIX{...}.
I know that in the original paper Cxx meant POISSON+FMIX{...}+G4, but the practical issues are quite compelling...

Best regards,
Stefan