IUC Contribution Fest - Metagenomics and mothur

[bgruening]

I will start this issue to list all idea specifically for all people that are interested in working on mothur during our Metagenomic Hackathon (#299).

I have created an initial branch with wrappers from @IyadKandalaft and team: https://github.com/galaxyproject/tools-iuc/tree/mothur

My idea is to use this issue to collect ideas specific to the mothur wrapper, coordinate efforts and create PR against this branch.

• improve datatypes and move them into Galaxy core
• evaluate tools from @yvanlebras http://toolshed.genouest.org/repository/view_repository?sort=name&operation=view_or_manage_repository&f-free-text-search=mothur&id=66701dbe65507c8c
• evaluate the usage of a data manager: https://github.com/fls-bioinformatics-core/galaxy-tools/tree/master/data_manager_mothur_toolsuite
• fix planemo lint issues
• Add test cases (Oksana)
• fix dataset collection integration
• improve usability as much as we can :)
• prepare some workflows to ship with the ToolShed and to demonstrate our achievements

Thanks everyone! This will be awesome!

[oxyko]

We have existing issue reported for this wrapper, so I'm pasting it here for everyone to be aware:

Problem with 2 of the Chimera tools (both in galaxy tool version 1.22 and 1.23) Chimera.slayer and Chimera.uchime.

In spite of the fact that in the drop down menu for the tool it states that a groups file is optional and a count file is optional instead of leaving these parameters out of the mothur command it uses "groups=None" and then the tool fails due to not being able to open file "None"

mothur > chimera.uchime(fasta=dataset_37011.dat,group=None,count=None,reference=self,abskew=1.9,minh=0.3,mindiv=0.5,xn=8.0,dn=1.4,xa=1.0,chunks=4,minchunk=64,idsmoothwindow=32,maxp=2,processors=8)
Unable to open None. Trying output directory None
Unable to open None. It will be disregarded.
Unable to open None. Trying output directory None
Unable to open None. It will be disregarded.
[ERROR]: no valid group files.
[ERROR]: The number of groupfiles does not match the number of fastafiles, please correct.

Using 8 processors.
[ERROR]: did not complete chimera.uchime.

mothur > chimera.slayer(fasta=dataset_37011.dat,reference=self,group=None,count=None,processors=8)
Unable to open None. Trying output directory None
Unable to open None. It will be disregarded.
Unable to open None. Trying output directory None
Unable to open None. It will be disregarded.
[ERROR]: no valid group files.
[ERROR]: The number of groupfiles does not match the number of fastafiles, please correct.

Using 8 processors.
[ERROR]: did not complete chimera.slayer.

[jmchilton]

I think the mothur datatypes should be modified so the short extension includes a prefix mothur. like mothur.otu.

[oxyko]

Will be installing and figuring out planemo and then working on the chimera tools issue and tests for them.

[jj-umn]

@jmchilton Agree on prefixing datatypes. Also, should see if any of these could just be subclassed, rather than python class. I guess that would depend on whether there is any useful metadata for the datatype.

[IyadKandalaft]

I'm looking at patching to support v1.36.1 of mothur

[bgruening]

Anyone up for a discussion round about mothur and the current state?
I have made an initial review of #429 and we can talk about it if you like.

[oxyko]

I've refactored all wrappers to use macros, as discussed earlier today. Currently macros takes care of requirements, citations and some common text from help. Wanted to extract the wrapper version as well, but the versions are all over the place: from 1.20.0 to 1.27.0. So not sure if it is better to use one version for all the wrappers, and if yes, then which one. Any thoughts?

[bgruening]

@oxyko this is a great work!

[shiltemann]

@oxyko awesome! :)

Also, since all files pretty consistently use 1-space indentation, I could do something like sed -i 's/^(\ *)/\1\1\1\1/g ' to all xml files to quadruple all leading spaces? ..or shall we we not worry about this for now?

[bgruening]

@shiltemann I think this is a good idea. I would wait before we have merged @oxyko PR to prevent more merge conflicts. But yes this is the way to go.

[bgruening]

@oxyko I think we should aim to target the latest release, right. So fix the latest release available in the requirement tag :)

[bgruening]

@shiltemann did a huge cleanup PR in #447
Please rebase your work to the latest changes. Thanks!
@oxyko you also need to rebase

[oxyko]

@shilteman Done! Thanks for clean up!

Also refactored the wrapper versions (all, but TreeVector.xml, which is another tool) to use one version from macros. The version is now set to 1.27.0. Same pull request (#448). Please merge.

[blankenberg]

Just a drive-by to +1 making Mothur paired dataset collection aware, especially for e.g. Make.contigs (and possibly others).

[shiltemann]

@blankenberg thanks, see #853

[martenson]

Closing this issue since this codefest is long finished.