mg7 v1.0.0-RC1

This is the first release candidate for v1.0.0. It is focused on refining 1.0.0-M5, fixing bugs and updating dependencies to stable versions.

• #124: Removed headers from the merged assignment tables
• #125: Fixed frequency count calculation bug
• #120: Added pipeline name as a prefix for the loquatID
• #123: Made default parameters presets extensible
• #121: Updated all dependencies

---

See the full list of pull requests merged in this release in the v1.0-RC1 milestone.

mg7 v1.0.0-M5

This milestone doesn't add many new features, but rather focuses on the internal code improvements and testing with a new reference database and several datasets. Here are the main changes since the v1.0-M4 release:

• #71, #113: BLAST output filtering based on the maximum pident is now configurable
• #112: Breaking changes to the way MG7 pipelines are defined and used in the user code: thanks to the improvements in Loquat user doesn't need to write boilerplate code to define an MG7 pipeline.
• #107, #63: Stats and Summary steps are removed from the pipeline
• #84, #96, #97, #98, #99: Global code review and refactoring
• #78: Switched to the CSV format for all tables
• #95: Bio4j taxonomy-related code is now in a separate project: ohnosequences/ncbitaxonomy
• #86, #102 : Added Illumina and Pacbio BEI mock communities as test pipelines

---

See the full list of pull requests merged in this release in the v1.0-M5 milestone.

mg7 v1.0.0-M4

This milestone adds some important features and improves the output data presentation. Here are the main changes since the v1.0-M3 release:

• New features:
  • #65: Added average pident (percentage of identity) column to the assignment and counts tables
  • #66: Combining multiple reference DBs
  • #68: Assignment now can handle multiple taxas per reference sequence
  • #72: BBH is now evaluated as the lowest common ancestor of the most rank-specific nodes of the max-bitscore hits
  • #70: Replaced db.rnacentral library dependency with the ReferenceDB class
• Various output tables improvements:
  • #69: Moved lineage column to the 1st position
  • #74: Added rank number for ordering rows by rank
  • #75, #76: Changed frequency counts to percentage (i.e. 100-based)

---

See the full list of pull requests merged in this release in the v1.0-M4 milestone.

mg7 v1.0.0-M3

This is the third milestone on the way to the v1.0 release. Main purpose of this release was to introduce some essential features and check consistency of the pipeline (primarily for Illumina data). Here are the main changes since the v1.0-M2 release:

• New features:
  • #9, #11: Added general stats for the standard pipeline (with Flash) (stats and summary steps)
  • #20: Added counts frequency tables
  • #35: Added taxonomic lineage in the results
  • #47: Changed to the fair lowest common ancestor (LCA) algorithm without the most-specific node modification
  • #56: Interchanged the order of the assign and merge steps to parallelize assignment (per BLAST chunk)
  • BLAST related improvements:
    • #46: Reads with no BLAST hits are accumulated per each sample and merged in FASTA files
    • #44: BLAST command itself and its parameters are completely configurable (so you can use not only blastn)
    • #50: Added a check for the set of minimal necessary BLAST output fields
    • #36, #51, #61: Reviewed and refined default BLAST parameters (for Illumina) and output format
    • #38, #49, #57, #58: Added configurable filter for the BLAST results (with default qcovs == 100%), plus an mandatory filter to keep only the top BLAST hits
• Other improvements:
  • #43, #53, ...: Upgraded to fastarious 0.6.0
  • #40, #41, ...: Upgraded to blast-api 0.7.0
  • #39: Upgraded to Loquat v2.0-M8
  • #53: Migrated to the RNACentral library for the reference DB interface
  • #11, #54, #59: Added outputs with possible losses in the assignment step (id2taxa mapping or Bio4j lookup)
• Important fixes:
  • #48: Configured CSV writer to use unix line-endings
  • #62: Fixed inconsistency in the accumulated counting algorithm
  • #52, #55: Fixes of the bugs introduced during M3 development

---

See the full list of pull requests merged in this release in the v1.0-M3 milestone.

mg7 v1.0.0-M3-fair-lca-top-hits

This intermediate release has two important features:

• #47: Fair LCA algorithm
• #58: A filter to keep only the top BLAST hits

Check the v1.0.0-M3 release for the full release notes.

mg7 v1.0.0-M2

This milestone release is huge. It contains a lot of improvements and some important fixes. Here are some higlights:

• Features and improvements:
  • #4, #15: The BLAST reference database and IDs mapping is now configurable
  • #17: Added RNACentral reference DB (as the default one now)
  • #18: BLAST options are now configurable
  • #16: A dataflow API was improved and a pipeline without the Flash step was added
  • #20 #24: For pipelines without Flash you can choose between FASTA and FASTQ input format
  • Output tables improvements:
    • #10: Added CSV-headers
    • #19, #23, #27: Added taxonomic rank and name columns
    • #6: Split each counts table on two: direct/accumulated
    • #30: Zero direct counts are filtered out
• Important fixes:
  • #28: Bio4j-NCBITaxonomy database was updated with an important fix for the taxonomic names
  • #26: Ongoing effort to solve the "Too many open files" exception has been moved to ohnosequences/loquat#56

See the full list of pull requests merged in this release in the v1.0-M2 milestone.

mg7 v1.0.0-M1

This is the first milestone release after testing the pipeline on real data. It's based on Loquat v2.0.0-M7.