Lmod is automatically replacing "intel/2020.1.217" with "gcc/9.3.0". Lmod is automatically replacing "intel/2020.1.217" with "gcc/9.3.0". The following have been reloaded with a version change: 1) hmmer/3.3.2 => hmmer/3.2.1 The following have been reloaded with a version change: 1) hmmer/3.2.1 => hmmer/3.3.2 You are running the following MDMcleaner command: '/home/ruizhang/.local/bin/mdmcleaner set_configs --db_basedir /project/def-careg421/ruizhang/software/mdmcleaner_db' reading settings from configfile: "/home/ruizhang/.local/lib/python3.11/site-packages/mdmcleaner/mdmcleaner.config" wrote settings to 'mdmcleaner.config' You are running the following MDMcleaner command: '/home/ruizhang/.local/bin/mdmcleaner clean -i /project/def-careg421/ruizhang/2021_guiyang/raw_bins/dastool/bins/metabat2_SK_surface_bin_.93_sub.fasta -o /scratch/ruizhang/mdmcleaner/test1 -c /project/def-careg421/ruizhang/software/mdmcleaner_db/mdmcleaner.config -t 32' reading settings from configfile: "/home/ruizhang/.local/lib/python3.11/site-packages/mdmcleaner/mdmcleaner.config" reading settings from configfile: "/project/def-careg421/ruizhang/software/mdmcleaner_db/mdmcleaner.config" settings: threads = '32' db_basedir = '['/project/def-careg421/ruizhang/software/mdmcleaner_db']' db_type = '['gtdb']' blacklistfile = '['/home/ruizhang/.local/lib/python3.11/site-packages/mdmcleaner/blacklist.list']' blastn = 'blastn' blastp = 'blastp' makeblastdb = 'makeblastdb' blastdbcmd = 'blastdbcmd' diamond = 'diamond' barrnap = 'barrnap' hmmsearch = 'hmmsearch' aragorn = 'aragorn' prodigal = 'prodigal' checking dependencies... blastp...2.13.0 --> OK! blastn...2.13.0 --> OK! diamond...2.1.6 --> OK! aragorn...1.2.38 --> OK! barrnap...0.9 --> OK! hmmsearch...3.3.2 --> OK! -->OK! All dependencies are being met! Database versions: GTDB version = r214 RefSeq release = release218 silva_download_dict = 138.1 ============================================================ Now processing genome: metabat2_SK_surface_bin_.93_sub -->doing ORF-calling subdividing contigs of /project/def-careg421/ruizhang/2021_guiyang/raw_bins/dastool/bins/metabat2_SK_surface_bin_.93_sub.fasta for multiprocessing divided 634 contigs into 32 fractions -->Detecting marker genes... searching prok_pfam.hmm ... searching prok_tigr.hmm ... searching prok_cog.hmm ... searching bact_tigr.hmm ... searching bact_pfam.hmm ... searching arch_pfam.hmm ... searching arch_tigr.hmm ... parsing protmarkerdict: 0 parsing protmarkerdict: 1 parsing protmarkerdict: 2 -->scanning for rRNA genes... found 1 rRNA sequences -->scanning for tRNAs... found 18 of 21 types of tRNAs --> estimated completeness = 90.00% (rough estimate based on 20 universally required types of tRNA) -->blasting protein data reading in protblast files... looking up taxids of protein blast hits... adding info to blastlines (old version) saving protblasts for reuse -->blasting rRNA data nothing to blast... nothing to do... looking up taxids of nucleotide blast hits... adding info to blastlines (old version) saving nuclasts for reuse -->LCA classifications classifying protein sequences... classified 100 records so far classified 200 records so far classified 300 records so far classified 400 records so far classified 500 records so far classified 600 records so far classified 700 records so far classified 800 records so far classified 900 records so far classified 1000 records so far classified 1100 records so far classified 1200 records so far classified 1300 records so far classified 1400 records so far classified 1500 records so far classified 1600 records so far classified 1700 records so far classified 1800 records so far classified 1900 records so far classified 2000 records so far classified 2100 records so far classified 2200 records so far classified 2300 records so far classified 2400 records so far classified 2500 records so far classified 2600 records so far classified 2700 records so far classified 2800 records so far classified 2900 records so far classified 3000 records so far classified 3100 records so far classified 3200 records so far classified 3300 records so far classified 3400 records so far finished classifying 3418 records! classifying rRNA sequences... finished classifying 0 records! -->determining major taxon majority tax-path: Cellular organisms; Bacteria; Bacteroidota; UBA10030; UBA10030; UBA10030; 2-02-FULL-55-14 completeness: 90.0 (based on universally required types of tRNA) -->evaluating contigs and setting filter-flags finished 6%. Currently analysing apparent reference database ambiguity finished 7%. Currently analysing apparent reference database ambiguity finished 11%. Currently analysing apparent reference database ambiguity finished 11%. Currently analysing apparent reference database ambiguity An error occured during blastp run with query '-' terminate called after throwing an instance of 'std::__ios_failure' what(): basic_ios::clear: iostream error THERE WAS A EXCEPTION WHILE HANDLING /project/def-careg421/ruizhang/2021_guiyang/raw_bins/dastool/bins/metabat2_SK_surface_bin_.93_sub.fasta multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/home/ruizhang/.local/lib/python3.11/site-packages/mdmcleaner/blasthandler.py", line 799, in run_single_blast blastcmd.check_returncode() File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.11.2/lib/python3.11/subprocess.py", line 502, in check_returncode raise CalledProcessError(self.returncode, self.args, self.stdout, subprocess.CalledProcessError: Command '['blastn', '-query', '-', '-db', '/project/def-careg421/ruizhang/software/mdmcleaner_db/gtdb/concat_refgenomes', '-evalue', '1e-10', '-outfmt', '6 std qlen slen', '-num_threads', '32', '-out', './tempdir_y3gcsv4s_mdmcleaner_refdbcontam/refblast_GCA_003153195.1_PLXG01000157.1__000blastn_auxblasts_vs_concat_refgenomes.tsv.tmp']' died with . During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.11.2/lib/python3.11/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) ^^^^^^^^^^^^^^^^^^^ File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.11.2/lib/python3.11/multiprocessing/pool.py", line 51, in starmapstar return list(itertools.starmap(args[0], args[1])) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ruizhang/.local/lib/python3.11/site-packages/mdmcleaner/blasthandler.py", line 883, in _run_any_blast outname = run_single_blastn(query, db, path_appl, outfmt, outname, threads) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ruizhang/.local/lib/python3.11/site-packages/mdmcleaner/blasthandler.py", line 812, in run_single_blastn return run_single_blast(query, db, blast, outfmt, outname, threads) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ruizhang/.local/lib/python3.11/site-packages/mdmcleaner/blasthandler.py", line 803, in run_single_blast raise RuntimeError RuntimeError """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/ruizhang/.local/lib/python3.11/site-packages/mdmcleaner/clean.py", line 197, in main db_suspects = bindata.evaluate_and_flag_all_contigs(db=db, protblasts=protblasts, nucblasts=nucblasts, db_suspects=db_suspects, fast_run=args.fast_run) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ruizhang/.local/lib/python3.11/site-packages/mdmcleaner/getmarkers.py", line 1199, in evaluate_and_flag_all_contigs is_potential_refdb_contam = step2() ^^^^^^^ File "/home/ruizhang/.local/lib/python3.11/site-packages/mdmcleaner/getmarkers.py", line 1166, in step2 db_suspects.parse_evidence(ambiguityinfo["amb_evidence"], self.contigdict[contig]["toplevel_marker"]) File "/home/ruizhang/.local/lib/python3.11/site-packages/mdmcleaner/review_refdbcontams.py", line 354, in parse_evidence self.evaluateornot(c) File "/home/ruizhang/.local/lib/python3.11/site-packages/mdmcleaner/review_refdbcontams.py", line 281, in evaluateornot evaluation, blastfilenames = comp.blast_contigs(self.threads, blacklist=self.blacklist, outfileprefix = os.path.join(self.tempdir, "")) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ruizhang/.local/lib/python3.11/site-packages/mdmcleaner/review_refdbcontams.py", line 70, in blast_contigs resultfiles = blasthandler.run_multiple_blasts_parallel(basic_blastarglist, outfmt= "6 std qlen slen", outbasename=outbasename, total_threads=threads) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ruizhang/.local/lib/python3.11/site-packages/mdmcleaner/blasthandler.py", line 927, in run_multiple_blasts_parallel outfile_list = masterblaster.starmap(_run_any_blast, arglist) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.11.2/lib/python3.11/multiprocessing/pool.py", line 375, in starmap return self._map_async(func, iterable, starmapstar, chunksize).get() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.11.2/lib/python3.11/multiprocessing/pool.py", line 774, in get raise self._value RuntimeError ------------------------------finished------------------------------