Possible issues with taxonony-mapping

[h836472]

Dear Diamond Community!

I am writing to ask for help while trying to perform query sequence taxon labeling based on a custom (Uniref-based) database.
Natively, the custom fasta header, that I would like to use, looks like this:

>UniRef100_Q6GZX3:t10493:Viruses Uncharacterized protein 002L n=3 Tax=Frog virus 3 TaxID=10493 RepID=002L_FRG3G

Linked to that, there is a tax map file that looks like the example below:

_accession.version{tab}taxid
UniRef100_Q6GZX3:t10493:Viruses{tab}10493

Using diamond version 2.1.4, the output of database creation is rather unusual:

Accessions in database 1813
Entries in accession to taxid file 1813
Yet there is 0 accession or sequence mapped to taxid...

db creation command:
/SSD/software/diamond/diamond-2.1.4/build/diamond makedb --in 10k_example.fasta --taxonmap 10k_example.taxmap --taxonnodes nodes.dmp --taxonnames names.dmp --db 10k_example

Then, when I perform a search against the database with the -f 102 format option, I only get 0 0 next to each query

UniRef100_Q6GZX3:t10493:Viruses 0 0

Realizing, that the custom fasta headers in the example are rather long and ugly, I repeated the entire procedure with a shortened version, where fasta headers were trimmed back to Accession number. Example:
>UniRef100_Q6GZX3
and the matching taxon mapping:
accession.version{tab}taxid
UniRef100_Q6GZX4{tab}10492

Upon the database generation, again I get 1813 sequences, with 1813 of them being present in taxid file and yet 0 accession to taxid mapping...

Search output is also exactly the same as above, 0 0 being printed next to every ID...

Can someone please help me, how I should perform taxon-enabled database generation and taxon labeling with diamond?

[bbuchfink]

When creating the mapping file, try to cut the UniRef100_ prefix from your accessions.

[h836472]

Dear Benjamin,

Thank you for the suggestion. After removing the prefix, as suggested, the small_scale test dataset (with ~10k lines) went trough all right. Database creation passed with all sequences properly linked to taxID. Also, subsequent "-f 102" search finished with a decent-looking output.

Then, I went back to the whole data that contains ~300M proteins with the exact same header layout. Did the trimming as suggested and tried database creation. Unfortunately, it failed with the following error message (last 6 lines pasted)

Writing accessions... [0.567s]
Hashing sequences... [0.108s]
Loading sequences... [0s]
Writing trailer... [6.741s]
Loading taxonomy names... [1.133s]
Loaded taxonomy names for 0 taxon ids.
Loading taxonomy mapping file... Error: Tokenizer Exception

I double-checked the taxon mapping and it looks OK. Can you please help me with this new problem too?

With kind regards,

Balazs

[bbuchfink]

I have added a line number to this error message to narrow down the problem: 40e440f (you have to clone the repository and compile from source).

[h836472]

Dear Benjamin,

Thank you for the data debugging capability added to your code. Indeed, line#191,411,926 of the taxon mapping is bogus:

UPI0000111246 10090
UPI0000111275 N/A
UPI00001114D9 9838

The format error goes back to the UniRef100 database where -occasionally- N/A is used as a "taxonID" indicating missing taxonomy information. I recall a communication with Uniprot staff about this format inconsistency, suggesting the use of taxid 32644 for such cases. I was in the belief that this error has already been cured out from my data but it looks like some N/A-s still managed to escape... Sorry for the confusion this error caused.

I shall get rid of any remaining N/A-s in the mapping file and give you an update about the database creation and the subsequent taxon labeling run.

With kind regards,

Balazs

[h836472]

Dear Benjamin,

After fully cleaning out the "N/A" values from the taxon mapping file, both the database generation as well as the taxon labeling with diamond went trough all right. Thank you for your help in tracing back the cause of the issue I encountered. End of the day, it was inappropriate data...