Bug: No way to avoid ERROR: Initial tree is not compatible with constraint tree

[corneliusroemer]

Thanks a lot for IQtree! I use it to make reference trees for Nextclade.

I use a pretty sophisticated constraint tree to avoid homoplasies pulling together branches I know should be separate.

I now get an error that's quite frustrating as there seems to be no clear workaround and it's almost impossible to debug giving limited logging: ERROR: Initial tree is not compatible with constraint tree

This feels like a bug as the initial tree should be built to be compatible with whatever constraint tree I pass in. There should be a way to build an initial tree just from parsimony addition to the constraint tree - that way the constraint would always be satisfied.

If that's hard to implement, at least better logging would be very useful: if you could output which part of the constraint tree is violated, and if it was possible to output the initial tree so that I could inspect in a treeviewer, that'd be super amazing.

Here are all the necessary files to reproduce.

Command I run:

$ iqtree2 -ntmax 1 -s builds/wuhan/masked_recombinant_XBB-delim.fasta \
    -m GTR -ninit 2 -n 2 -me 0.05 -nt AUTO\
    -redo -czb -ninit 1 -n 1 -g profiles/clades/constraint_XBB.nwk \
    > builds/wuhan/masked_recombinant_XBB-delim.iqtree.log
OMP: Info #276: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
ERROR: Initial tree is not compatible with constraint tree

Files are in here (just get rid of .txt ending and untar, the txt is necessary to be able to upload here): constraintbug.tzst.txt

Version:

$ iqtree2 -v                                                                                          
IQ-TREE multicore version 2.2.2.3 COVID-edition for Mac OS X 64-bit built May 16 2023

From logs:

Reading constraint tree profiles/clades/constraint_XBB.nwk...
    665 WARNING: Rooted constraint tree will be treated as unrooted tree
    666 Checking for duplicate sequences: done in 0.00272107 secs using 665.7% CPU
    667 Creating fast initial parsimony tree by random order stepwise addition...
    668 3.655 seconds, parsimony score: 1610 (based on 1212 sites)
    669 ERROR: Initial tree is not compatible with constraint tree

[roblanf]

Hi Cornelius,

@bqminh @thomaskf and I have a meeting today and I'll bring this up. Hopefully we can get a solution to you quickly!

Rob

[corneliusroemer]

Thanks a lot for the quick response! I think I somehow managed to workaround the issue for now. So it's not urgent. I wrote a script that tells me which splits in the parsimony tree (found it in outdir) are not in the constraint tree (~600 terminals) and found something. When I pruned that split, things worked again. Not 100% sure that that was it. It's entirely possible I'm somehow feeding in a bad tree that makes no sense and that it's just the log/error output that needs to be clearer. In general, more verbose output for errors like this would be super useful. When you have big trees, manual inspection is a hopeless task, so if Iqtree could spit out more of it's internal state when something errored that would be amazing.

…

On Tue, Sep 26, 2023, 23:12 roblanf ***@***.***> wrote: Hi Cornelius, @bqminh <https://github.com/bqminh> @thomaskf <https://github.com/thomaskf> and I have a meeting today and I'll bring this up. Hopefully we can get a solution to you quickly! Rob — Reply to this email directly, view it on GitHub <#144 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF77AQJWCN6ZZ3AZWELHOILX4NANDANCNFSM6AAAAAA5HZVLDI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[corneliusroemer]

Just hit this issue again. I wish I had copied that script here. I'll give you full repro details again for testing in case someone has a look at this part of the code:

iqtree2 -ntmax 1 -s builds/wuhan/masked_recombinant_XBB-delim.fasta -m GTR \
 -ninit 2 -n 2 -me 0.05 -nt AUTO -redo -czb \
 -ninit 1 -n 1 -g builds/wuhan/pruned_constraint_XBB.nwk \
> builds/wuhan/masked_recombinant_XBB-delim.iqtree.log

IQ-TREE multicore version 2.2.5 COVID-edition for Mac OS X 64-bit built Sep 15 2023

All input/output files are in this archive:
parstreebug.tar.zst.gz

Edit: I found the script and added it as a Gist here for others to use in case they also encounter this error: https://gist.github.com/corneliusroemer/0b49aac048e6ba404388ba0ae829691e

My script doesn't find any disagreement between constraint tree and parsimony tree, strange. It's of course possible the script is wrong:

python scripts/constraint_detector.py builds/21L/pruned_constraint_XBB.nwk builds/21L/masked_recombinant_XBB-delim.fasta.parstree 
[]

[corneliusroemer]

Hitting this issue again, new tree, again can't find any disagreement between provided constraint tree and parsimony tree. Now v2.2.6

 iqtree2 -ntmax 4 -s builds/wuhan/masked_masked_without_recombinants-delim.fasta -m GTR -ninit 2 -n 2 -me 0.05 -nt AUTO -redo -czb -g builds/wuhan/pruned_constraint.nwk -ninit 1 -n 1 > builds/wuhan/masked_masked_without_recombinants-delim.iqtree.log

(txt extension is so I can upload it here)
constraintbug-2024-01-11.tar.zst.txt

Update: Upon closer inspection I found an issue in my constraint tree. I had labelled an internal node JN.1.2 but that was also a real sequence given to the tree builder. Hence the incompatibility, I presume.

Constraint looked something like this:

                (
                  BA.2.86.1,
                  (
                    JN.1,
                    (
                      JN.1.1,
                      JN.1.1.3
                    )JN.1.2,  <--- problem is here, there's a comma missing before JN.1.2

[thomaskf]

Hi @corneliusroemer,
Sorry for the delay of the reply. I have tried two of your datasets "constraintbug-2024-01-11.tar.zst.txt" and "constraintbug.tzst.txt" by using the version 2.3.4. IQ-TREE2 can finish smoothly. The issue should have been resolved. Thanks for reporting the issue.