Issue with retrieving sequences from the CAZy database

[kataszl203]

Hi!

I've encountered an issue when attempting to retrieve sequences from the CAZy database using the following command:

cw_get_genbank_seqs <path to local CAZy database> <user email address>

Initially, I successfully downloaded the CAZy database using the command:

cazy_webscraper  <user email address> --families AA9

Followed by:

cw_get_uniprot_data <path to local CAZy database> --pdb --sequence --families AA9

These steps proceeded without any errors. However, upon executing the cw_get_genbank_seqs command, I encountered the following issue for many, if not all, records:

"Could not retrieve a NCBI protein version accession matching an accession from the local database from the record id 'CAA34853.1'. The sequence from this record will not be added to the db."

Additionally, I received the following traceback:

Traceback (most recent call last):
  File "/cazy_env/bin/cw_get_genbank_seqs", line 8, in <module>
    sys.exit(main())
  File "/cazy_env/lib/python3.6/site-packages/cazy_webscraper/expand/genbank/sequences/get_genbank_sequences.py", line 164, in main
    args,
  File "/cazy_env/lib/python3.6/site-packages/cazy_webscraper/expand/genbank/sequences/get_genbank_sequences.py", line 321, in get_seqs_from_ncbi
    args,
  File "/cazy_env/lib/python3.6/site-packages/cazy_webscraper/expand/genbank/sequences/get_genbank_sequences.py", line 534, in parse_failed_connections
    failed_connections_batches["_".join(batch)]
KeyError: 'U_U_A_0_5_2_9_4_._1'

As the following step I tried using the cw_extract_db_seqs command:

cw_extract_db_seqs <path to local CAZy database> genbank uniprot --fasta_dir /fasta/ --families AA9

Which resulted in another traceback:

Traceback (most recent call last):
  File "/cazy_env/bin/cw_extract_db_seqs", line 8, in <module>
    sys.exit(main())
  File "/cazy_env/lib/python3.6/site-packages/cazy_webscraper/expand/extract_seqs/extract_db_seqs.py", line 160, in main
    extracted_sequences.update(get_uniprot_sequences(uniprot_table_dict, gbk_dict))
  File "/cazy_env/lib/python3.6/site-packages/cazy_webscraper/expand/extract_seqs/extract_db_seqs.py", line 262, in get_uniprot_sequences
    gbk_id = uniprot_table_dict[uniprot_accession]['genbank_id']
KeyError: 'genbank_id'

I would be immensely grateful if you could provide some insights or guidance on how to resolve these issues.

Thank you very much for your time and assistance.
Kasia

[HobnobMancer]

Hi!

Thanks for you using cazy_webscraper, and sorry its not working for you.

That's definetly a bug in cazy_webscraper not parsing the NCBI protein version accession correctly. I've created an issue #129 for this and will hopefully get it sorted in the next week or so.

As the cw_get_genbank_seqs failed, there is not point in running cw_extract_db_seqs to extract the GenBank sequences as there are not GenBank sequences in your local CAZyme database (that stage failed). I'll have a look at this once I have the GenBank sequence retrieved working again 👍

[HobnobMancer]

Sorry it's taken so long to get round to this. I have started work on PR #130 - this includes significantly reducing the memory requirements for cazy_webscraper.

Hopefully, this PR will merged into the main branch in the next week or two 🤞

[HobnobMancer]

I am still working on this. I've been on holiday recently. But I hope have working code on the issues-129-ncbi branch but next Monday 🤞