install_NCBITaxa.py | ete3 NCBI taxonomy database

[RhettRautsaw]

Error

When running
python MITGARD/install_NCBITaxa.py

>>> ncbi.update_taxonomy_database()
Downloading taxdump.tar.gz from NCBI FTP site...
Done. Parsing...
Loading node names...
2294790 names loaded.
233217 synonyms loaded.
Loading nodes...
2294790 nodes loaded.
Linking nodes...
Tree is loaded.
Updating database: /home/rrautsa/.etetoolkit/taxa.sqlite ...
 2294000 generating entries... 
Uploading to /home/rrautsa/.etetoolkit/taxa.sqlite

Inserting synonyms:      90000 Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/rrautsa/.conda/envs/mitosis_env/lib/python3.6/site-packages/ete3/ncbi_taxonomy/ncbiquery.py", line 113, in update_taxonomy_database
    update_db(self.dbfile)
  File "/home/rrautsa/.conda/envs/mitosis_env/lib/python3.6/site-packages/ete3/ncbi_taxonomy/ncbiquery.py", line 740, in update_db
    upload_data(dbfile)
  File "/home/rrautsa/.conda/envs/mitosis_env/lib/python3.6/site-packages/ete3/ncbi_taxonomy/ncbiquery.py", line 779, in upload_data
    db.execute("INSERT INTO synonym (taxid, spname) VALUES (?, ?);", (taxid, spname))
sqlite3.IntegrityError: UNIQUE constraint failed: synonym.spname, synonym.taxid

Explanation

MitoZ will not work if you receive this error! The reason this error occurs is due to the fact that the NCBI taxonomy database used by MitoZ is constantly updating. The latest updates do not work well with ete3.

Solution

This is unlikely to be a problem in the future, once ete3 updates. However, for now, this can be solved by downloading an older version of the taxdump database and manually running the code inside install_NCBITaxa.py with slight modification...

First download an older database and set it as a tar.gz file.
In bash:

# bash
cd ~
mkdir taxdump
cd taxdump
wget ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/taxdmp_2020-01-01.zip
unzip taxdmp_2020-01-01.zip
rm *.zip
tar --remove-files -cvzf taxdump.tar.gz * 

Then in python:

#python
from ete3 import NCBITaxa
ncbi = NCBITaxa()
ncbi.update_taxonomy_database('taxdump.tar.gz')
quit()