JOSS Publication next steps

[kescobo]

I've now completed the addition of all of the primary features that I wanted for Microbiome.jl before submitting to Joss,

• Add dictionary methods to CommunityProfiles #74
• Add cladefilter #77
• Add generic filter for CommunityProfiles #79

So I'd like to discuss next steps that we can accomplish over the next 2 weeks, with the intent that we'll be finished with code / tutorials by October 11th, and submitting a manuscript by the end of October. @annelle-abatoni @anikaluo I am not sure how much you're intending to work over the next month, but this is going to have to move pretty fast, and so I'd like to try to get a realistic sense of how much you'll be able to do.

We can start individual discussions for all of these things, but here's the big items as I see it:

1. Add features to BiobakeryUtils.jl. The big ones are Make sure all DataFrames - returning functions switch to comm profile BiobakeryUtils.jl#30, Plotting BiobakeryUtils.jl#25, adding rm_strat BiobakeryUtils.jl#47, and a few of the things from I/O For other biobakery outputs BiobakeryUtils.jl#23. Most important from that last, I'd really like to get something that will take a profile + metadata file, and make a CommunityProfile, or that will take a CommunityProfile and write two CSVs.
2. A few outstanding issues in Microbiome.jl Change add_metadata! for single samples to set! and insert! #75 Add docs for Dictionaries methods #76
3. Tutorials for Microbiome.jl and BiobakeryUtils.jl
4. Documentation cleanup

1 - BiobakeryUtils.jl features

Part of me thinks that I should just do these to make things go fast, but I want your feedback. Do you want to try to get some of these things written yourself? If so, can you commit to working on it and iterating quickly after you get my feedback?

2 - Outstanding issues in Microbiome.jl

Similar story here. Though I think these will be pretty straightforward

3 - Tutorials

This is the big one, and the place I think the two of you can make the biggest difference in terms of my effort / focus.

Microbiome.jl

The microbiome.jl tutorial should basically have a narrative form that steps through the use of:

1. The Taxon and GeneFunction feature types
2. How to construct them
3. various accessor methods (clade, hasclade, name, etc)
4. The MicrobiomeSample type
5. especially how metadata works (various Dictionaries.jl methods)
6. The CommunityProfile type
   1. How to build one
   2. How to get the various parts out (abundances, samples, features)
   3. How to update sample metadata
   4. The Tables.jl interface (for both abundances and metadata)

BiobakeryUtils.jl

I think this should be divided up into a MetaPhlAn tutorial and a HUMAnN tutorial

1. For both, mention that it assumes you're using version 3 of the tool
2. For Metaphlan
3. Be sure to link to the metaphlan tutorial
4. Most of the things that are done in this section of the tutorial to explore the contents of a profile can be done with BiobakeryUtils.jl and Microbiome.jl
5. Similarly, show how to do some of the plots here (this might be something I need to do)
6. Similar strategy for humann

4 - Documentation cleanup

• Make sure all functions and types are documented (and that the documentation shows up when the docs build)
• Make sure all doctests pass and are actually checked
• Make sure that docs cross-reference eachother using @ref (eg CommunityProfile here should actually link to the docstring of CommunityProfile
• Probably other stuff too

[anikaluo]

questions / comments / concerns:

• What does “Owrite” mean?
  • “with the intent that we'll be finished with code / tutorials by Owrite and submit a manuscript by the end of October 11th”
• How much I’ll be able to do = optimistically ~5-6 hours a week?
• Package features: Depending on how complex, I think it would be cool to write things ourselves, but that would definitely take longer for everyone because we would spend longer than you do writing things and you would have to check them. Also, with classes and everything else during the day I’m not sure how quickly I can iterate.
• Tutorials: I think we can definitely do these.
• I can totally try making plots for BiobakeryUtils.jl! But only if you think it is doable for me.
• I haven’t actually worked on Microbiome.jl, but I can definitely familiarize myself with it.
• (Probably more that I haven't thought of yet)

[kescobo]

1. "Owrite" was a typo - fixed now. Want to be done with code and documentation and tutorials by October 11th, submit by end of October
2. Not a lot, but probably documentation and tutorial reading / running / editing
3. Fair
4. great!
5. Especially given time, I think I'll do plots - help / editing would be appreciated

[kescobo]

FYI - now thanks to EcoJulia/BiobakeryUtils.jl#65, you can now do BiobakeryUtils.install_deps() to install humann and metaphlan on your system. This also makes it possible to test the command line functionality in the test suite (see here)

[kescobo]

If you're going to do any additional docs review before our meeting tomorrow, please checkout main branches of both pages, and make new branches from there to make PRs or issues (note, if you just want to open issues, you can link to a specific line of code A lot has changed over the last couple of days.

Current version of Microbiome docs: http://biojulia.net/Microbiome.jl/dev/

Current version of BiobakeryUtils docs: http://biojulia.net/BiobakeryUtils.jl/dev/

I made some big changes to indexing CommunityProfiles in #87 and #85, so getting a double-check that the indexing examples in the humann and metaphlan tutorials would be really useful. I'm going to do some work on other stuff today, and then work on the manuscript tomorrow

[anikaluo]

Do you want us to keep reviewing the tutorials or wait until tomorrow's meeting?

[kescobo]

If you've got time today, yes - just do it from the current main branch. And I've already made some changes to BiobakeryUtils.jl here: EcoJulia/BiobakeryUtils.jl#71