v0.3.2

Bug-fix release, from an independent review of the 0.3.1 changes.

Fixes a 0.3.1 regression

• Symlinked fastq files were rejected. 0.3.1's new "fastq files in subfolders" check followed each file's symlink, so an input folder of symlinks to reads stored elsewhere failed — a layout that worked in 0.3.0 and that QIIME2 imports fine. If you hit "QIIME2 imports fastq files from the top level of the input folder only" on 0.3.1, upgrade.

Metadata

• #q2:missing is honored (INSDC:missing terms such as not applicable are missing values, not a group). Such a column could read as eligible and then fail at beta-group-significance.
• A UTF-8 BOM (Excel export) in front of a comment line no longer makes every column disappear.
• Duplicated sample ids or column names, and columns whose names collide once made file-name-safe (Host Plant / Host_Plant), are rejected up front instead of after DADA2.
• beta-group-significance is best-effort per column, like the sample classifiers: one column QIIME2 refuses no longer aborts a finished run.

Other

• Archive extraction works on Python patch releases older than 3.9.17 / 3.10.12 / 3.11.4.
• A report built from a run_metadata.json with blanked user/host prints "(not recorded)".
• Validation suite: checks outputs, not just exit codes (it would have caught the ≤0.3.0 classifier bug); refuses to wipe a directory it didn't create. New dated validation record with an erratum on the previous one.

Reminder from 0.3.1: classifiers trained with qiime2-its-train-ncbi / qiime2-its-train-fasta on ≤ 0.3.0 should be retrained.

327 unit tests, CI on Python 3.9–3.13, 8 end-to-end scenarios on QIIME2 2026.7.

v0.3.1

Bug-fix release from a full code review.

Retrain your classifiers

qiime2-its-train-ncbi and qiime2-its-train-fasta kept only one reference sequence per taxid (every other sequence of the same species was silently dropped from taxonomy.txt). Classifiers trained with 0.3.0 or earlier should be retrained. qiime2-its-train-unite was not affected.

Pipeline

• Fails fast, before any work: missing metadata/classifier file, fastq samples with no metadata row, output folder inside the input folder, fastq files in subfolders (without -rc). Previously these surfaced after DADA2 or as cryptic QIIME2 importer errors.
• -t/--threads is now honored by the phylogeny and taxonomy-classification steps (both used every core).
• Metadata is read with QIIME2's own rules (# comment rows, quoted cells); empty cells are missing values, not a group — a column with one real group plus blanks could previously crash the run at beta-group-significance.

Report

• Columns with spaces/special characters (e.g. Host Plant) keep their real name and their groups in PCoA/dendrogram pages and tables.
• Alpha boxplots share group colors with the PCoA/dendrogram; samples with no value are shown as "(missing)" in grey.
• Numeric-looking sample IDs (001) keep matching the metadata.

Trainers

• train-ncbi: a crashed download can no longer be mistaken for a finished one; an empty NCBI result stops before the multi-GB downloads.
• train-fasta: every fasta ID is checked against the id table up front.
• Archives are detected by content; a non-archive raises a clear error. Downloads have a socket timeout.

Misc

• qiime2-its-fastq-rc refuses an output folder inside its input folder.
• Durations no longer print as 1m60s.

Validated end to end against the bundled real datasets (8 scenarios) on QIIME2 2026.7; 309 unit tests, CI on Python 3.9–3.13.

v0.3.0

A minor release built around the PDF report: adds new diagnostic figures, expands which metadata
columns get their own diversity plots, and overhauls the report's visual design — plus several bug
fixes found by generating and visually inspecting real reports rather than just running the test
suite.

Added

• UPGMA sample-clustering dendrogram next to each PCoA page, built from the same distance
  matrix core_diversity already computes — the standard complement to a PCoA scatter for beta
  diversity, added without any new QIIME2 step.
• Sequence-length distribution and classification-confidence distribution figures — two
  quick QC signals (over/under-trimmed ITS extraction; how well the classifier resolved this run's
  ASVs) using artifacts the pipeline already produces.
• Alpha/beta diversity figures for every statistically significant metadata column, not just
  one: previously only --report-metadata-column's column got a boxplot/PCoA/dendrogram, even
  though the underlying significance tests already covered every eligible column. Now any column
  with p < 0.05 gets its own figures too, organized into one sub-section per column (both distance
  metrics' PCoA and dendrogram together) with a note explaining why each column appears.
• train_ncbi.py now fingerprints the query (file content, for an accession-list query) alongside
  seq.fasta, so a different query reusing the same -o output folder is detected and
  re-downloaded instead of silently training on stale sequences.
• Active-conda-env cross-check for -q/--qiime2 in pipeline.py, matching train_unite.py.

Changed

• Report-wide visual redesign: switched from an 8-color palette to CARTOColors "Safe" (11
  colorblind-safe hues, verified against CartoColor's own source), with marker-shape cycling on the
  PCoA scatter once a column has more groups than the palette has colors. Publication-quality
  tables (three-line style, zebra striping), a short explanatory paragraph per report section, and
  wider/taller figures tuned for real (>10-sample) datasets.
• --report-metadata-column is now the report's always-shown default grouping column rather than
  its only one — any other significant column also gets figures (see Added, above).

Fixed

• Several real report-rendering bugs, each found by regenerating and visually inspecting an actual
  53-sample report rather than assuming the code was correct: overlapping run-info labels, a PCoA
  legend sitting on top of data points, an alpha-boxplot category (oil seep) fully hidden behind
  its rotated 45-degree neighbors' labels, a legend long enough to squeeze its own plot into half
  the page, and group colors that weren't consistent between a column's PCoA and dendrogram pages.
• 6 latent bugs found by a full code-audit pass, including a dead -q/--qiime2 flag in
  train_unite.py that was recorded for provenance but never actually cross-checked against the
  active environment.

Dependencies

• Added scipy (UPGMA clustering for the new dendrogram figure).

v0.2.2

A patch release focused on real-data robustness and PyPI-readiness, prompted by running the pipeline against a real published dataset (Okrasińska et al. 2022, Environmental Microbiology — 53 real ITS2/UNITE soil-fungal samples, SRA BioProject PRJNA767765) end to end for the first time at production scale.

Fixed

• Casava filename validation rejected sample identifiers containing dots (common in real SRA-derived names, e.g. K.BeL.1.1) — a valid file was rejected outright before the pipeline did anything else.
• Sample-frequency parsing choked on QIIME2's comma thousands-separator formatting once a sample's frequency reached four digits (e.g. "110,406.0") — broke the PDF report on any real dataset with more than ~1000 reads/sample, which every prior test dataset was too small to trigger.

Both were found and fixed against the real failure case, not just in isolation — the run that hit each bug went on to complete successfully afterward.

Changed (PyPI packaging prep)

• Fixed README links (logo, license) that were relative to the GitHub repo and would 404 on PyPI's project page.
• Modernized license metadata to the SPDX form (license = "MIT" + license-files), replacing the table form setuptools was warning would stop being supported.
• Added [project.urls] (Homepage/Repository/Documentation/Issues/Changelog) for PyPI's project-page sidebar.
• Added CITATION.cff (enables GitHub's "Cite this repository").

Documentation

• Documented Eukaryome as an additional, directly-fetchable ITS reference source alongside UNITE (qiime rescript get-eukaryome-data).
• Documented why SILVA/GTDB/PR2 aren't usable for ITS classification (none cover the ITS region), including SILVA's pretrained classifier downloads for anyone using this pipeline on a non-ITS marker instead.
• README quick start now leads with installing/activating QIIME2 itself instead of assuming it's already set up.

v0.2.1

A patch release: a new run-provenance/QA feature for report.pdf, a new logo, a real report-rendering bug fix, and a couple of small install/report polish items.

New: run provenance / QA metadata

Every run now writes <output>/run_metadata.json (even with --skip-report), and report.pdf gains four new pages near the front for it:

• Run information: start/end time, duration, user@host, platform, conda environment, qiime2-its/QIIME2 framework/BBMap versions, input/metadata/classifier/output paths, and the exact command invoked (formatted as a readable, one-flag-per-line block).
• Pipeline parameters: every CLI flag's value, for exact reproducibility.
• Input sample files: one row per input fastq file, grouped by sample.
• Installed QIIME2 plugins: every plugin qiime info reports, with version.

A report built from an older output folder that predates this just skips those pages rather than failing.

Fixed

• The beta-diversity group-significance table in report.pdf could overlap adjacent columns' text for longer metadata column names (e.g. host-plant) — split into clean metadata column/distance metric columns, plus a general truncate-with-ellipsis backstop so an unexpectedly long value can't cause this again elsewhere in the report.

Changed

• New logo (circular badge design) in the README, wiki, and report.pdf's title page.
• Installation now installs BBTools/BBMap alongside ITSxpress in a single conda install command, instead of as a separate, easy-to-skip later step.
• The wiki's source now lives in this repo (wiki/, published via scripts/sync_wiki.sh), so wiki edits get the same review/diff/history as code.

v0.2.0

A full rewrite of the pipeline: installable package, unit + real-data test coverage, updated for current QIIME2, and a substantial new feature set on top of the original denoising workflow.

Breaking changes

• Replaces the old flat scripts (python qiime2_its.py ...) with an installable package (pip install -e .) exposing five console-script entry points: qiime2-its, qiime2-its-rc, qiime2-its-train-unite, qiime2-its-train-ncbi, qiime2-its-train-fasta. Command-line flags are otherwise unchanged.
• ITS region extraction now goes through the qiime itsxpress plugin instead of the standalone itsxpress CLI (the standalone CLI is gone as of ITSxpress 2.x). Install with conda install -c bioconda -c conda-forge itsxpress && qiime dev refresh-cache inside your QIIME2 environment.
• Targets current QIIME2 (rachis-qiime2-2026.7) — several qiime subcommands changed their flags since this pipeline was last updated (see below); older QIIME2 versions are no longer supported.

New: diversity stats, sample classification, and a PDF report

Runs by default after the core pipeline (opt out with --skip-advanced-stats/--skip-report):

• qiime diversity alpha-group-significance / beta-group-significance (Kruskal-Wallis / PERMANOVA) against every eligible metadata column.
• qiime taxa collapse (genus level, capped to whatever depth the classifier actually resolved) + relative-abundance table.
• qiime sample-classifier classify-samples (random forest) per eligible column, skipped per-column with a clear reason rather than aborting the run when a column can't support it.
• report.pdf: a lightweight PDF summary (run parameters, DADA2 retention, alpha/beta group-significance tables and plots, genus-level composition, rarefaction curve, classifier accuracy) built with matplotlib + fpdf2 — no LaTeX/browser-rendering/system dependencies.

New: qiime2-its-train-fasta

Trains a classifier from your own FASTA + accession/taxid table, alongside the existing UNITE and NCBI training paths.

Testing and validation

• 150+ unit tests (mock subprocess.run, assert the constructed command — no QIIME2 installation needed to run them); CI on every push.
• A committed, real-data validation suite (validation/) that actually runs QIIME2 against small real datasets (paired-end, single-end, IonTorrent-style, empty/near-empty samples, multi-sample, UNITE/NCBI/FASTA classifier training) and records results as a dated audit trail in validation/results/.
• Several real QIIME2-2026.7 compatibility breaks and pipeline bugs found and fixed via that validation — see validation/results/*/SUMMARY.md for the full, dated list (denoising-stats output requirements, feature-table summarize flag changes, classify-sklearn threading flag semantics, a taxonomy rank-mapping bug, a corrupted test fixture, and others).

Documentation

• README trimmed to a quick install/run; full docs (installation, building a classifier, pipeline usage, metadata/fastq requirements, the advanced-stats/report feature, the validation suite, development, troubleshooting) moved to the wiki, now tracked as source in wiki/ in this repo (scripts/sync_wiki.sh publishes it).
• New logo and CI/license/version badges.
• Pipeline-usage docs now include concrete DADA2 tuning recommendations for single-end IonTorrent data, and note which IonTorrent-relevant DADA2 options (HOMOPOLYMER_GAP_PENALTY, BAND_SIZE) QIIME2's plugin doesn't expose.

v0.1.2 - Minor improvements

Improved the "train classifier" scripts to accept files as input.

v0.1.1 - bug fix

Minor fixes/New features
Fixed an issue where the remove_empty_fastq_entries.py script couldn't be located.
Added an option to choose the number of parallel processing. The pipeline would crash if user was entering a number of threads smaller than 4 because a hard coded value of 4 parallel was used.
Checks were added to handle the number of threads and parallel processes properly.

v0.1 - Initial release

Initial release.