Error During findorfs, the following command failed with return code 2:

[SabeenR]

Hello,
So I've been trying to work through the various errors that get generated while running the ./run_pipline.test.sh

I'm attaching 3 files:
1 - FINDORFS-Errors.pdf
2 - RUNPIPELINE.log.pdf
3 - FINDORFS.log.pdf

• Why did the command FINDORFs fail ?
• some of the files it's looking for does exisit yet it says it doesn't) please note the the directory lists I've copied at the bottom of the FINDORFs-Errors file.

How can I resolve this ?

Thanks a bunch
Sabeen

[skoren]

First, you can ignore errors in run pipeline, those are cleanup steps to ensure intermediate failed outputs aren't in the way of a current run. metAMOS will ignore those failed commands and keep going.

Have you modified run_pipeline_test.sh or are you using a different test script? run_pipeline_test.sh would not use Prokka. The only way to enable Prokka is to run with --isolate mode (iMetAMOS) (test cases like test_ima.sh or test_gage.sh)

That said, prokka should still work on this sample. The error in FindORFS is coming from one of the Prokka subcommands. That binary (rnammer) is not packaged with Prokka and not downloaded by metAMOS so it must be installed in your path somewhere and is producing an error. It could be a version incompatible with Prokka, I would suggest removing it from your path and trying again.

[SabeenR]

I think the problem was that I added the -W imetamos flag at the end of the initPipeline.sh line in the run_pipeline_test.sh. I removed it and ran it and it went through to the Postprocess stage now.

But it still doesn’t give me the summary.html file in the Postprocess/out/html folder.

The Postprocess/out/html folder has the following:
Abundance.html
Annotate.html
asmstats.out
symb link to class.classified
plot.tab
propogate.in.clusters
propogate.out.clusters

Does that still mean it worked ?

From: Sergey Koren [mailto:notifications@github.com]
Sent: Wednesday, September 28, 2016 1:20 PM
To: marbl/metAMOS metAMOS@noreply.github.com
Cc: Raza, Sabeen sraza@bcm.edu; Author author@noreply.github.com
Subject: Re: [marbl/metAMOS] Error During findorfs, the following command failed with return code 2: (#250)

_CAUTION:_ This email is not from a BCM Source. Only click links or open attachments you know are safe.

---

First, you can ignore errors in run pipeline, those are cleanup steps to ensure intermediate failed outputs aren't in the way of a current run. metAMOS will ignore those failed commands and keep going.

Have you modified run_pipeline_test.sh or are you using a different test script? run_pipeline_test.sh would not use Prokka. The only way to enable Prokka is to run with --isolate mode (iMetAMOS) (test cases like test_ima.sh or test_gage.sh)

That said, pork should still work on this sample. The error in FindORFS is coming from one of the Prokka subcommands. That binary (rnammer) is not packaged with Prokka and not downloaded by metAMOS so it must be installed in your path somewhere and is producing an error. It could be a version incompatible with Prokka, I would suggest removing it from your path and trying again.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_marbl_metAMOS_issues_250-23issuecomment-2D250253793&d=DQMFaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=xtM6GPkkLyIR9otuERCXkg&m=KjSMFhl-fvv88qkJvl7CeYboF7Z_JwVlAVaNoLoPx54&s=tctcZNB8LW4x24LWhm4F8LwQn6CXJoEJU1Vvn8KN1mM&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AVXI4Rk4wpo1PPrIPAJPVuPC2MvT99vyks5quq-5FOgaJpZM4KJEC4&d=DQMFaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=xtM6GPkkLyIR9otuERCXkg&m=KjSMFhl-fvv88qkJvl7CeYboF7Z_JwVlAVaNoLoPx54&s=juu3NtWvBHYfMcabgop2oRdHJLx8wbSVu2tpW3meGRE&e=.

[skoren]

Did you make sure you have perl's Statistics::Descriptive package installed (see issue #248)

[SabeenR]

Oh yes, yes I did thanks .. I installed it using CPAN

That error flag is gone now.. PHEW !

I also did download the DataAnalysis package from Roche which has newbler in it but I cannot figure out where to put the symbolic link to the executable so that the newbler warning goes away as well. The only one left then will be SignalP+ but I’m assuming both tese are optional and not critical for metAMOS or iMetAMOS to run correctly.

I’m running runPipeline.py right now with a small genome that we’ve sequenced. Keeping fingers crossed.

I’ll let you know what happens once it’s run through.

Thanks for all your help & advice.

Sabeen

From: Sergey Koren [mailto:notifications@github.com]
Sent: Wednesday, September 28, 2016 3:30 PM
To: marbl/metAMOS metAMOS@noreply.github.com
Cc: Raza, Sabeen sraza@bcm.edu; Author author@noreply.github.com
Subject: Re: [marbl/metAMOS] Error During findorfs, the following command failed with return code 2: (#250)

_CAUTION:_ This email is not from a BCM Source. Only click links or open attachments you know are safe.

---

Did you make sure you have perl's Statistics::Descriptive package installed (see issue #248https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_marbl_metAMOS_issues_248&d=DQMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=xtM6GPkkLyIR9otuERCXkg&m=RLEmaZlPBovuqEXV8-rgX39EMl6z4PKoAArOxxdcEvo&s=_oHHqM6Jj1Hl5usKguNCZRkAoL2jng0EPWvP8UlAWSI&e=)

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_marbl_metAMOS_issues_250-23issuecomment-2D250290028&d=DQMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=xtM6GPkkLyIR9otuERCXkg&m=RLEmaZlPBovuqEXV8-rgX39EMl6z4PKoAArOxxdcEvo&s=jTa0IklbEJdNwlSxvysU76IDWhVrMi3RZjw6lxgJJMw&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AVXI4ZpCCKVTOqDCIV6qwUkkY6EyW0Ncks5qus44gaJpZM4KJEC4&d=DQMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=xtM6GPkkLyIR9otuERCXkg&m=RLEmaZlPBovuqEXV8-rgX39EMl6z4PKoAArOxxdcEvo&s=KoODUPP3jEGg2kjRv6TZsLz-Y6UIci9ZIdN8I9HinQQ&e=.

[SabeenR]

OK the process was completed but with errors:

The Postprocess/out/html directory has the following in it:

[root@tcmc_sandbox2 metAMOS-1.5rc3]# ls -l /home/jspinler/iMetAMOS_initpipeline/Postprocess/out/html/
total 720
-rw-r--r--. 3 root root 2122 Sep 28 15:25 Abundance.html
-rw-r--r--. 3 root root 14411 Sep 28 15:19 Annotate.html
-rw-r--r--. 1 root root 309 Sep 28 15:45 asmstats.out
lrwxrwxrwx. 2 root root 49 Sep 28 15:45 class.classified -> /home/jspinler/iMetAMOS_initpipeline/Classify/out
-rw-r--r--. 1 root root 68 Sep 28 15:45 plot.tab
-rw-r--r--. 2 root root 289099 Sep 28 15:25 propagate.in.clusters
-rw-r--r--. 2 root root 414396 Sep 28 15:26 propagate.out.clusters

*** metAMOS running command: ln /home/jspinler/iMetAMOS_initpipeline/FunctionalAnnotation/out/ec.krona.html /home/jspinler/iMetAMOS_initpipeline/Postprocess/out/html/FunctionalAnnotation.html

*** metAMOS running command: perl -I /home/AnalysisTools/metAMOS-1.5rc3/AMOS/Linux-x86_64/lib /home/AnalysisTools/metAMOS-1.5rc3/Utilities/perl/statistics.pl /home/jspinler/iMetAMOS_initpipeline/Postprocess/out/proba.scf.fa > /home/jspinler/iMetAMOS_initpipeline/Postprocess/out/html/asmstats.out

Oops, MetAMOS finished with errors! see text in red above for details.
Traceback (most recent call last):
File "./runPipeline", line 985, in
verbose = 1)
File "/home/AnalysisTools/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 2965, in pipeline_run
raise job_errors
RethrownJobError:

Exception #1
  'exceptions.TypeError(histogram() got an unexpected keyword argument 'new')' raised in ...
   Task = def postprocess.Postprocess(...):
   Job  = [proba.asm.contig -> proba.scf.fa]

Traceback (most recent call last):
  File "/home/AnalysisTools/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 625, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/home/AnalysisTools/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 491, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "/home/AnalysisTools/metAMOS-1.5rc3/src/postprocess.py", line 286, in Postprocess
    create_summary("%s/Abundance/out/%s.taxprof.pct.txt"%(_settings.rundir,_settings.PREFIX),"%s/Postprocess/out/%s.bnk"%(_settings.rundir,_settings.PREFIX),"%s/Postprocess/out/html/"%(_settings.rundir),"%s/Postprocess/out/%s.scf.fa"%(_settings.rundir,_settings.PREFIX),"%s"%(_settings.METAMOS_UTILS),"%s/img"%(_settings.METAMOSDIR),"%s"%(_settings.AMOS),len(_readlibs),"%s"%(_settings.taxa_level),"%s"%(_settings.DB_DIR))
  File "/home/AnalysisTools/metAMOS-1.5rc3/Utilities/python/create_summary.py", line 234, in create_summary
    create_plots("%s/plot.tab"%(html_prefix),"%s"%("proba1"))
  File "/home/AnalysisTools/metAMOS-1.5rc3/Utilities/python/create_plots.py", line 165, in create_plots
    n, bins, patches = plt.hist(h,  100)
  File "/usr/lib64/python2.7/site-packages/matplotlib/pyplot.py", line 2657, in hist
    stacked=stacked, **kwargs)
  File "/usr/lib64/python2.7/site-packages/matplotlib/axes.py", line 8101, in hist
    m, bins = np.histogram(x[i], bins, weights=w[i], **hist_kwargs)
TypeError: histogram() got an unexpected keyword argument 'new'

What did I do wrong ?

From: Sergey Koren [mailto:notifications@github.com]
Sent: Wednesday, September 28, 2016 3:30 PM
To: marbl/metAMOS metAMOS@noreply.github.com
Cc: Raza, Sabeen sraza@bcm.edu; Author author@noreply.github.com
Subject: Re: [marbl/metAMOS] Error During findorfs, the following command failed with return code 2: (#250)

_CAUTION:_ This email is not from a BCM Source. Only click links or open attachments you know are safe.

---

Did you make sure you have perl's Statistics::Descriptive package installed (see issue #248https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_marbl_metAMOS_issues_248&d=DQMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=xtM6GPkkLyIR9otuERCXkg&m=RLEmaZlPBovuqEXV8-rgX39EMl6z4PKoAArOxxdcEvo&s=_oHHqM6Jj1Hl5usKguNCZRkAoL2jng0EPWvP8UlAWSI&e=)

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_marbl_metAMOS_issues_250-23issuecomment-2D250290028&d=DQMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=xtM6GPkkLyIR9otuERCXkg&m=RLEmaZlPBovuqEXV8-rgX39EMl6z4PKoAArOxxdcEvo&s=jTa0IklbEJdNwlSxvysU76IDWhVrMi3RZjw6lxgJJMw&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AVXI4ZpCCKVTOqDCIV6qwUkkY6EyW0Ncks5qus44gaJpZM4KJEC4&d=DQMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=xtM6GPkkLyIR9otuERCXkg&m=RLEmaZlPBovuqEXV8-rgX39EMl6z4PKoAArOxxdcEvo&s=KoODUPP3jEGg2kjRv6TZsLz-Y6UIci9ZIdN8I9HinQQ&e=.

[skoren]

For newbler, you can add whatever direction runProject contains to your path. You can also create a folder in metAMOS named newbler/Linux-amd64/ and link newAssembly, addRun, and runProject in that folder.

Your other error is an incompatible matplotlib. metAMOS requires matplotlib 1.3.0, newer versions don't work. You need to remove your system matplotlib from your python environment and let metAMOS install it's own or install 1.3.0 yourself. Alternatively, you'd have to edit the metAMOS create_plots.py code to remove the histogram creation (essentially lines 157-257)

[SabeenR]

Thank you so very much Sergey.

I removed the metplotlib directory from the python path and it WORKED !! no more errors ☺

From: Sergey Koren [mailto:notifications@github.com]
Sent: Wednesday, September 28, 2016 4:39 PM
To: marbl/metAMOS metAMOS@noreply.github.com
Cc: Raza, Sabeen sraza@bcm.edu; Author author@noreply.github.com
Subject: Re: [marbl/metAMOS] Error During findorfs, the following command failed with return code 2: (#250)

_CAUTION:_ This email is not from a BCM Source. Only click links or open attachments you know are safe.

---

For newbler, you can add whatever direction runProject contains to your path. You can also create a folder in metAMOS named newbler/Linux-amd64/ and link newAssembly, addRun, and runProject in that folder.

Your other error is an incompatible matplotlib. metAMOS requires matplotlib 1.3.0, newer versions don't work. You need to remove your system matplotlib from your python environment and let metAMOS install it's own or install 1.3.0 yourself. Alternatively, you'd have to edit the metAMOS create_plots.py code to remove the histogram creation (essentially lines 157-257)

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_marbl_metAMOS_issues_250-23issuecomment-2D250308774&d=DQMFaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=xtM6GPkkLyIR9otuERCXkg&m=yQUTVCk5LhzVDH-rTkVzVNX0y-XwucqVbW9zagbwb8k&s=vYKWpk1P4ML4-MOIGHahhwy4_AwGNu4xFyVz5ax3ZlQ&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AVXI4QBbf1lOX0tJse69HuNca5j660Xqks5qut6LgaJpZM4KJEC4&d=DQMFaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=xtM6GPkkLyIR9otuERCXkg&m=yQUTVCk5LhzVDH-rTkVzVNX0y-XwucqVbW9zagbwb8k&s=yUKTDv4HBaXG2Ij6c9GzlrGnZQJ5UdarUqHeC6vWPPs&e=.