Error when using bsub

[ssscj]

Hi, thanks for developing this program. I ran C3POa on login nodes, and it worked. But I used bsub to submit the task to computing nodes, it was stuck at the step of calling consensi.

Aligning splints to reads with blat
Preprocessing: 100%|██████████| 10/10 [00:06<00:00,  1.59it/s]
Catting psls: 100%|██████████| 10/10 [00:00<00:00, 86.01it/s]
Removing preprocessing files: 100%|██████████| 10/10 [00:00<00:00, 287.00it/s]
Calling consensi:   0%|          | 0/10 [00:00<?, ?it/s]

the command I used is this:
bsub -q TEST-A -n 32 -e c3pao.err -o c3poa.out 'python ~/software/c3poa/C3POa/C3POa.py -r ../01.QC/nanofilt/rep1/test.fastq -o output_test/ -s ../10x_UMI_splint.fasta -c c3pao.config -l 1000 -d 500 -n 32 -g 1000'

Do you know how to run C3POa by submitting bsub command? Thank you!

Best regards,
Chujie

[rvolden]

I've never used bsub, but I have a question about your default python version? Are you trying to run this with python 2? Most servers still have python 2.7 as the default python, which might subtly break things. You can try running explicitly with python3 and maybe that will help? If it doesn't, try changing line 235-238 from

pool.apply_async(analyze_reads,
                args=(args, tmp_reads, splint_dict, adapter_dict, adapter_set, iteration, racon),
                callback=lambda _: pbar.update(1)
)

to

pool.apply_async(analyze_reads,
                args=(args, tmp_reads, splint_dict, adapter_dict, adapter_set, iteration, racon),
                callback=lambda _: pbar.update(1)
).get()

Error messages don't get passed up through apply_async, so if there's something erroring, adding the .get() should let you see those error messages at the cost of multiprocessing.

[ssscj]

Hi, thanks for your reply.The server uses python 3.7. I changed the code but it did not give any error message. Luckily it worked successfully on another server which does not use bsub.