BCALM 2, git commit 7ca49a8 setting storage type to hdf5 [Approximating frequencies of minimizers ] 100 % elapsed: 1 min 12 sec remaining: 0 min 0 sec cpu: 100.0 % mem: [ 31, 31, 31] MB [DSK: nb solid kmers found : 2462748976 ] 100 % elapsed: 43 min 40 sec remaining: 0 min 0 sec cpu: 877.4 % mem: [4167, 9148, 9204] MB bcalm_algo params, prefix:SRR1298980.unitigs.fa k:41 a:4 minsize:10 threads:96 mintype:1 DSK used 1 passes and 486 partitions prior to queues allocation 12:28:28 memory [current, maxRSS]: [ 644, 9204] MB Starting BCALM2 12:28:28 memory [current, maxRSS]: [ 644, 9204] MB [Iterating DSK partitions ] 0 % elapsed: 0 min 0 sec remaining: 0 min 0 sec Iterated 947015 kmers, among them 64397 were doubled In this superbucket (containing 1034 active minimizers), sum of time spent in lambda's: 17650.6 msecs longest lambda: 213.9 msecs tot time of best scheduling of lambdas: 17650.6 msecs best theoretical speedup: 82.5x Done with partition 0 12:28:29 memory [current, maxRSS]: [1464, 9204] MB [Iterating DSK partitions ] 9.05 % elapsed: 0 min 37 sec remaining: 6 min 8 sec Iterated 2104511 kmers, among them 175968 were doubled Loaded 56514 doubled kmers for partition 49 In this superbucket (containing 530 active minimizers), sum of time spent in lambda's: 16642.2 msecs longest lambda: 146.3 msecs tot time of best scheduling of lambdas: 16642.2 msecs best theoretical speedup: 113.7x Done with partition 49 12:29:09 memory [current, maxRSS]: [1781, 9204] MB [Iterating DSK partitions ] 19.8 % elapsed: 1 min 29 sec remaining: 6 min 3 sec Iterated 3460329 kmers, among them 374870 were doubled Loaded 202591 doubled kmers for partition 98 In this superbucket (containing 907 active minimizers), sum of time spent in lambda's: 26076.7 msecs longest lambda: 140.4 msecs tot time of best scheduling of lambdas: 26076.7 msecs best theoretical speedup: 185.8x Done with partition 98 12:30:00 memory [current, maxRSS]: [1988, 9204] MB [Iterating DSK partitions ] 29.6 % elapsed: 2 min 25 sec remaining: 5 min 44 sec Iterated 2987639 kmers, among them 408521 were doubled Loaded 289181 doubled kmers for partition 147 In this superbucket (containing 959 active minimizers), sum of time spent in lambda's: 23597.7 msecs longest lambda: 158.1 msecs tot time of best scheduling of lambdas: 23597.7 msecs best theoretical speedup: 149.3x Done with partition 147 12:30:56 memory [current, maxRSS]: [2152, 9204] MB [Iterating DSK partitions ] 39.5 % elapsed: 3 min 17 sec remaining: 5 min 2 sec Iterated 3022030 kmers, among them 551928 were doubled Loaded 460592 doubled kmers for partition 196 In this superbucket (containing 1225 active minimizers), sum of time spent in lambda's: 24221.8 msecs longest lambda: 177.0 msecs tot time of best scheduling of lambdas: 24221.8 msecs best theoretical speedup: 136.9x Done with partition 196 12:31:50 memory [current, maxRSS]: [2248, 9204] MB [Iterating DSK partitions ] 50.2 % elapsed: 4 min 25 sec remaining: 4 min 23 sec Iterated 5509498 kmers, among them 631364 were doubled Loaded 411765 doubled kmers for partition 245 In this superbucket (containing 766 active minimizers), sum of time spent in lambda's: 44148.7 msecs longest lambda: 570.6 msecs tot time of best scheduling of lambdas: 44148.7 msecs best theoretical speedup: 77.4x Done with partition 245 12:32:55 memory [current, maxRSS]: [2734, 9204] MB [Iterating DSK partitions ] 60.1 % elapsed: 6 min 9 sec remaining: 4 min 5 sec Iterated 4741892 kmers, among them 415873 were doubled Loaded 274930 doubled kmers for partition 294 In this superbucket (containing 232 active minimizers), sum of time spent in lambda's: 36868.4 msecs longest lambda: 420.4 msecs tot time of best scheduling of lambdas: 36868.4 msecs best theoretical speedup: 87.7x Done with partition 294 12:34:41 memory [current, maxRSS]: [3416, 9204] MB [Iterating DSK partitions ] 69.1 % elapsed: 7 min 51 sec remaining: 3 min 30 sec Iterated 9198783 kmers, among them 858952 were doubled Loaded 767933 doubled kmers for partition 343 In this superbucket (containing 509 active minimizers), sum of time spent in lambda's: 74165.6 msecs longest lambda: 561.5 msecs tot time of best scheduling of lambdas: 74165.6 msecs best theoretical speedup: 132.1x Done with partition 343 12:36:37 memory [current, maxRSS]: [3484, 9204] MB [Iterating DSK partitions ] 79 % elapsed: 9 min 50 sec remaining: 2 min 37 sec Iterated 8715373 kmers, among them 903230 were doubled Loaded 1103877 doubled kmers for partition 392 In this superbucket (containing 779 active minimizers), sum of time spent in lambda's: 69051.3 msecs longest lambda: 301.9 msecs tot time of best scheduling of lambdas: 69051.3 msecs best theoretical speedup: 228.7x Done with partition 392 12:38:39 memory [current, maxRSS]: [3560, 9204] MB [Iterating DSK partitions ] 90.5 % elapsed: 12 min 11 sec remaining: 1 min 16 sec Iterated 10022315 kmers, among them 1246584 were doubled Loaded 2053772 doubled kmers for partition 441 In this superbucket (containing 2582 active minimizers), sum of time spent in lambda's: 77634.9 msecs longest lambda: 176.2 msecs tot time of best scheduling of lambdas: 77634.9 msecs best theoretical speedup: 440.7x Done with partition 441 12:40:43 memory [current, maxRSS]: [3571, 9204] MB [Iterating DSK partitions ] 99.6 % elapsed: 13 min 40 sec remaining: 0 min 3 sec Number of sequences in glue: 312283407 Number of pre-tips removed : 0 Buckets compaction and gluing : 820.3 secs Within that, creating buckets from superbuckets: 540.1 secs bucket compaction (wall-clock during threads): 263.4 secs within all bucket compaction threads, adding nodes to subgraphs: 7481.7 secs subgraphs constructions and compactions: 8947.8 secs compacted nodes redistribution: 3179.4 secs Sum of CPU times for bucket compactions: 20149.0 secs BCALM total wallclock (excl kmer counting): 820.5 secs Maximum number of kmers in a subgraph: 1043413 Performance of compaction step: Wallclock time spent in parallel section : 263.4 secs Best theoretical speedup in parallel section : 150.0x Best theor. speedup in parallel section using 1 threads : 1.0x Sum of longest bucket compaction for each sb : 148.9 secs Sum of best scheduling for each sb : 19609.1 secs Done with all compactions 12:42:13 memory [current, maxRSS]: [2803, 9204] MB bglue_algo params, prefix:SRR1298980.unitigs.fa k:41 threads:96 Starting bglue with 96 threads 12:42:13 memory [current, maxRSS]: [2803, 9204] MB number of sequences to be glued: 312283407 12:42:13 memory [current, maxRSS]: [2803, 9204] MB 182157650 marked kmers, 78073794 unmarked kmers 12:44:51 memory [current, maxRSS]: [3456, 9204] MB created vector of hashes, size approx 1389 MB) 12:44:51 memory [current, maxRSS]: [3456, 9204] MB pass 1/3, 91073053 unique hashes written to disk, size 694 MB 12:45:09 memory [current, maxRSS]: [2845, 9204] MB 182158736 marked kmers, 78073794 unmarked kmers 12:47:50 memory [current, maxRSS]: [3633, 9204] MB created vector of hashes, size approx 1389 MB) 12:47:50 memory [current, maxRSS]: [3633, 9204] MB pass 2/3, 91073728 unique hashes written to disk, size 694 MB 12:48:07 memory [current, maxRSS]: [2847, 9204] MB 182176634 marked kmers, 78073794 unmarked kmers 12:50:46 memory [current, maxRSS]: [3649, 9204] MB created vector of hashes, size approx 1389 MB) 12:50:46 memory [current, maxRSS]: [3649, 9204] MB pass 3/3, 91082577 unique hashes written to disk, size 694 MB 12:51:03 memory [current, maxRSS]: [2847, 9204] MB loaded all unique UF elements (273229358) into a single file vector of size 2084 MB 12:51:06 memory [current, maxRSS]: [4932, 9204] MB [Building BooPHF] 100 % elapsed: 0 min 7 sec remaining: 0 min 0 sec Bitarray 1286968192 bits (100.00 %) (array + ranks ) final hash 0 bits (0.00 %) (nb in final hash 0) UF MPHF constructed (153 MB) 12:51:13 memory [current, maxRSS]: [3000, 9204] MB