Skip to content

Datasets

Wenyu (Eddy) Huang edited this page May 28, 2026 · 3 revisions

Datasets

Three dataset tiers, each testing a different aspect of profiler behavior. Manifests live in data/datasets_*.txt (one fastq path per line) and are passed to the profiler driver via --dataset-list.

Tier Tests Ground truth Manifest
Simulated algorithm under known composition exact (simulated) data/datasets_simulated.txt
Mock (D6331) real sequencing of a defined community manufacturer composition data/datasets_mock_pacbio.txt, data/datasets_mock_ont.txt
Clinical (DYN) real-world inter-method concordance none (Illumina baseline as proxy) cohort cache (dyn_load_all.py)

Simulated — MIMIC + PBSIM3

PacBio and ONT reads simulated from a gut profile fitted with MIMIC and generated by PBSIM3 (data/simulated/{pacbio,ont}/sim_*.fastq); truth is the PBSIM3 reference taxon list. Used by Fig 8.

Both platforms are simulated from the same composition, so at simulated coverage sketch tools recover identical taxa regardless of platform error model — this tier tests algorithmic capability, not ONT-vs-PacBio error transfer.

Mock — ZymoBIOMICS Gut Microbiome Standard D6331

A defined community on both platforms; truth is the manufacturer composition. Used by Figs 2–3 (detection) and Figs 4–7 (abundance).

PacBio HiFi (data/datasets_mock_pacbio.txt) — shared Shoreline Breaker extraction, differing input mass:

ONT (data/datasets_mock_ont.txt) — two libraries differing only in DNA-extraction chemistry: D6331_Zymo (Zymo Quick-DNA) and D6331_Qiagen (Qiagen PowerSoil Pro), from the ZymoBIOMICS data portal.

Clinical — DYN (DYNAMITE cohort)

Antibiotic-perturbed gut metagenomes on PacBio + ONT with matched Illumina baselines. No defined truth, so the analysis reports cross-tool / cross-platform agreement. Used by Fig 9 (heatmaps) and Fig 10 (alpha-diversity). Cohort cache built by scripts/analysis/dyn_load_all.py.

Next steps

Clone this wiki locally