BLAST failures at usegalaxy.org

[jennaj]

End-user Workaround

NCBI BLAST+ tools had some corrections today (12/7/20). Full testing is still in progress but so far have turned out well.
Even if your combination of inputs is not marked as "pass" yet in the testing matrix below,
consider a rerun at this time anyway.

For any reruns, be sure to use the most current version of all tools in the BLAST+ tool suite (2.10.1+galaxy0).
This includes NCBI BLAST+ makeblastdb Make BLAST database (Galaxy Version 2.10.1+galaxy0)

Problematic tools and errors

• MTS: https://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus/5edc472ec434
• Dev repo: https://github.com/peterjc/galaxy_blast/tree/master/tools/ncbi_blast_plus
• See galaxy-bugs report for details/examples
• Some of those reported errors would fail for other reasons (size/memory/input-format) but the jobs never get far enough along to hit those usage issues
• protein or nucleotide, same failure
• unrelated to tool form options, whether appropriate for the input fasta or not (parse seqIDs in NCBI format yes/no; create seqID hash yes/no)
• input fasta pre-formatting (aka NormalizeFasta yes/no) doesn't matter either

ncbi_makeblastdb -- Fixing this will likely resolve some other problems that are server-related

• Failure due to some dependency issue eg "shared libraries: libssl.so.1.0.0" not available at roundup?

Bug report info
tool id: toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_makeblastdb/2.10.1+galaxy0
tool version: 2.10.1+galaxy0
job pid or drm id: XXXX
job tool version: makeblastdb: error while loading shared libraries: libssl.so.1.0.0: cannot open shared object file: No such file or directory

job stdout:

job info:
None

job traceback:
None

Job Information
Galaxy Tool ID: | toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_makeblastdb/2.10.1+galaxy0
Galaxy Tool Version: | 2.10.1+galaxy0
Tool Version: | makeblastdb: error while loading shared libraries: libssl.so.1.0.0: cannot open shared object file: No such file or directory
Tool Standard Output: | stdout
Tool Standard Error: | stderr
Tool Exit Code: | 127
Job Messages | Fatal error: Exit code 127 ()

Destination Parameters
Runner | slurm
Runner Job ID | XXXX
Handler | main_w3_handler0
docker_sudo | true
nativeSpecification | --partition=normal,jsnormal --nodes=1 --cpus-per-task=1 --time=36:00:00

ncbi_blastp_wrapper Other wrappers will also fail when using a blastdb created with ncbi_makeblastdb/0.3.0.

• Failing due to running on Stampede? (no cvmfs ??)
• AND/OR because cannot consume a user created blastdb ?
  • Users must create custom blastdbs using an earlier version of ncbi_makeblastdb to get this far along due to problems with the current version. Eg: database created using toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_makeblastdb/0.3.0.
  • The job failure below uses a custom blastdb created with v0.3.0.
• using a fasta file directly works, but the fasta must be really really small plus the stats are off.
  • Tool form warning You can also search against a FASTA file of subject (target) sequences. This is not advised because it is slower (only one CPU is used), but more importantly gives e-values for pairwise searches (very small e-values which will look overly signficiant). In most cases you should instead turn the other FASTA file into a database first using makeblastdb and search against that.

Bug report info
tool id: toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_blastp_wrapper/0.2.01
tool version: 0.2.01
job pid or drm id: XXXX
job tool version: blastp: 2.5.0+
Package: blast 2.5.0, build Sep 28 2016 01:45:10

job stderr:
BLAST Database error: No alias or index file found for protein database [/scratch/XXXX/xcgalaxy/main/staging//XXXX/inputs/dataset_XXXX_files/blastdb] in search path [/scratch/XXXX/xcgalaxy/main/staging/XXXX/working::]

job stdout:
LibCvmfs version 2.4, revision 25

job info:
None

job traceback:
None

Job Information
Galaxy Tool ID: | toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_blastp_wrapper/0.2.01
Galaxy Tool Version: | 0.2.01
Tool Version: | blastp: 2.5.0+ Package: blast 2.5.0, build Sep 28 2016 01:45:10
Tool Standard Output: | stdout
Tool Standard Error: | stderr
Tool Exit Code: | 2
Job Messages | Fatal error: Exit code 2 ()Fatal error:

Destination Parameters
Runner | pulsar_stampede
Runner Job ID | XXXX
Handler | main_w3_handler2
default_file_action | remote_transfer
dependency_resolution | remote
docker_sudo | true
file_action_config | /srv/galaxy/main/config/pulsar_stampede_actions.yml
jobs_directory | /scratch/XXXX/xcgalaxy/main/staging/
remote_metadata | true
remote_property_galaxy_datatypes_config_file | /cvmfs/main.galaxyproject.org/galaxy/config/datatypes_conf.xml.sample
remote_property_galaxy_home | /cvmfs/main.galaxyproject.org/galaxy
rewrite_parameters | true
submit_native_specification | --partition=normal --nodes=1 --ntasks=64 --time=48:00:00 --account=XXXX
submit_user_email | $user_email
transport | curl
use_remote_datatypes | true

ping @natefoo @davebx

[jennaj]

Test histories:

https://usegalaxy.org/u/jen-galaxyproject/h/test-blast

https://usegalaxy.eu/u/jenj/h/test-blast

[jennaj]

Tests:

#makeblastdb-2.10.1+galaxy0

• #target-accession-genomic #parse-seq-no == pass
• #target-ncbi-coding #parse-seq-no == pass
• #target-ncbi-coding #parse-seq-yes == pass
• #target-ncbi-coding-protein #parse-seq-yes == pass
• #target-ncbi-coding-protein #parse-seq-no == pass

#databaseinfo-2.10.1+galaxy0

• #target-accession-genomic #makeblastdb-2.10.1+galaxy0 #parse-seq-no == pass
• #target-ncbi-coding #makeblastdb-2.10.1+galaxy0 #parse-seq-no == pass
• #target-ncbi-coding #makeblastdb-2.10.1+galaxy0 #parse-seq-yes == pass
• #target-ncbi-coding-protein #makeblastdb-2.10.1+galaxy0 #parse-seq-yes == pass
• #target-ncbi-coding-protein #makeblastdb-2.10.1+galaxy0 #parse-seq-no == pass

---

• fail > dep issue > blastn: error while loading shared libraries: libssl.so.1.0.0: cannot open shared object file: No such file or directory`

#blastn-2.10.1+galaxy0

• #target-accession-genomic #makeblastdb-2.10.1+galaxy0 #parse-seq-no #exact-match-query-genomic == fail
• #target-ncbi-coding #makeblastdb-2.10.1+galaxy0 #parse-seq-no #exact-match-query-coding == fail
• #target-ncbi-coding #makeblastdb-2.10.1+galaxy0 #parse-seq-yes #exact-match-query-coding == fail

#dc-megablast-2.10.1+galaxy0

• #target-accession-genomic #makeblastdb-2.10.1+galaxy0 #parse-seq-no #exact-match-query-genomic == fail
• #target-ncbi-coding #makeblastdb-2.10.1+galaxy0 #parse-seq-no #exact-match-query-coding == fail
• #target-ncbi-coding #makeblastdb-2.10.1+galaxy0 #parse-seq-yes #exact-match-query-coding == fail

---

#blastp-2.10.1+galaxy0

• #target-ncbi-coding-protein #makeblastdb-2.10.1+galaxy0 #parse-seq-yes #exact-match-query-coding-prot == fail

---

#tblastn-2.10.1+galaxy0

• #target-ncbi-coding #makeblastdb-2.10.1+galaxy0 #parse-seq-yes #exact-match-query-coding-prot == fail

---

#blastx-2.10.1+galaxy0

• #target-ncbi-coding-protein #makeblastdb-2.10.1+galaxy0 #parse-seq-yes #exact-match-query-coding == fail

---

#tblastx-2.10.1+galaxy0

• #target-ncbi-coding #makeblastdb-2.10.1+galaxy0 #parse-seq-yes #exact-match-query-coding == fail

---

#blastdbcmd-2.10.1+galaxy0

• pending mapping results

---

[jennaj]

Needs to be tested again @jennaj (after the server updates to Galaxy v 21.01).

[jennaj]

Retests in progress. Tools appear to be working so far.

[jennaj]

All works, closing.

Note: People are having trouble when indexing fasta datasets that contain description content on the title lines, instead of just identifiers. Indexing is putatively successful, then mapping against it fails. We could possibly add more help to the makeblastdb tool form to explain why that format is important (>identifier only, no spaces or extra content). It is the same issue that custom genomes run into -- except the errors show up sooner with BLAST (during mapping, not after when downstream tools are used that are attempting to match up identifiers between inputs).