Data package data-v1.20.0

Microbiome Repo — data package v1.20.0 (2026-10-12)

One catalog, two tiers, all ages. The registry tables (registry_*) cover every public human shotgun-metagenome study in ENA/SRA/DDBJ — 64,369 studies, all body sites and ages — with archive-only classification (host, UBERON-anchored body sites, life stages, assay, access) and the harvested BioSample tier (637,397 BioSamples with normalised site / age / sex / country). The catalog tables (gut_*) are the curated tier: every registry study of the human gut, all ages — 2,946 studies, 602,163 samples, 4,216,259 sample × field values — with age at collection, sex, BMI, country, health condition (controlled vocabulary), antibiotic exposure, subject and timepoint recovered from archive attributes (R1), supplementary tables (R2), paper full text (R3) and abstracts (R4); every value carries a verbatim evidence quote (≤ 12 words), a labelled source, a route and a confidence. Coverage over all gut samples: age 28%, sex 33%, country 91%, health condition 49%, antibiotic exposure 20%.

Start here. gut_studies.parquet (one row per study: classification, papers, per-field coverage, age-category and health-condition mix, curated depth) → gut_sample_metadata_wide.parquet (one row per sample: every field with __route / __confidence, age_category + basis, body_site_class) → gut_sample_determinations.parquet (the evidence behind every value). registry_studies.parquet is the registry; registry_runs_v1.20.0.parquet (every run) ships as a GitHub Release asset.

Infant extension. The 389 infant-gut studies were curated in greater depth in earlier releases (delivery mode, feeding mode, preterm status, gestational age, birth weight, maternal antibiotics, probiotics, HMO supplementation, NEC). Those rows are present in the gut_* tables (curated_source = infant_catalog, infant_scope = true reproduces the former infant catalog exactly: 72,358 samples) and the original deep tables (sample_metadata_wide, sample_determinations, study_metadata_wide, cohorts, universe_studies_all, sandpiper_*, …) remain in the package for reproducibility; the paragraphs and headline numbers of that extension are kept below under Infant extension (historical description).

Infant extension (historical description)

The registry_* tables cover every public human shotgun-metagenome study in ENA/SRA/DDBJ (all body sites, all ages; archive-only classification plus the harvested BioSample tier). All other tables belong to the first curated scope, the Infant Gut Shotgun-Metagenome Catalog — the description below is that scope's.

A curated, evidence-linked catalog of every public shotgun-metagenome study of the human infant gut (0–36 months) in the INSDC archives (ENA/SRA/DDBJ), with per-sample metadata recovered from archive attributes, supplementary tables and papers.

Headline numbers (computed from the tables in this package; see CHANGELOG v1.2 and v1.2.2). 389 included studies · 154,206 catalog samples (153,685 BioSample units + 521 run units from 6 one-BioSample-per-infant deposits; their 16 parent BioSamples are listed in parent_biosamples.parquet and counted nowhere else) · 174,022 sequencing runs · 618,898 sample × field determinations, each with a verbatim evidence quote (9,559 of them are out-of-scope ages that document adult_age_flag) · age scope: 67,598 samples with per-sample or per-subject infant evidence + 7,097 in studies triage judged all-infant = 74,695 age-scope infant samples, of which 72,358 are also gut/unknown body site = catalog_scope, the headline denominator (2,900 age-scope infant rows are body-site excluded/linked: skin, milk, oral, mother…); 14,410 samples sit in mixed-age deposits and 49,556 in studies without a per-study infant estimate (age unknown); 9,559 carry an out-of-range age (adult_flagged); 5,986 are mother/other samples · age at collection ≤ 1,100 d recovered for 53,669 of 74,695 age-scope infant samples (72%) for 52,649 of 72,358 catalog-scope samples (73%) and for 62,230 of 146,336 body-site infant-scope samples (43%; v1.2.0 said 54,105 of 146,631 before the v1.2.2 host/isolate exclusions; the v1.1 figure 53,802 was computed before the run-unit fix and the site's 54,150 included the 16 parent rows and rows later superseded) · precision vs curatedMetagenomicData (hi-res set, 3,498 gold rows; extraction_gold_eval_hires.csv): age 0.99, delivery mode 0.995, preterm status 1.00 (full set of 3,670 rows: age 0.976, delivery 0.995, preterm 1.00 — see tier_field_precision.csv, column gold_set).

Universe screened: 9,579 candidate studies (all METAGENOMIC WGS/WXS runs in ENA with a human signal, no taxon filter) → 389 included, 52 unresolvable after two full-text review rounds; recall on the 22 curatedMetagenomicData infant BioProjects: 17/22 by the automated cascade, 18/22 after the human-review pass (one study, decision_stage=session_model_review), the rest excluded with a coded reason (universe_studies_all.parquet).

Catalog membership: a study is in the catalog when it is in gut_studies.parquet (samples: gut_sample_metadata_wide.parquet). The catalog_status / triage_verdict / reason_code columns of universe_studies_all.parquet and study_metadata_wide.parquet are the historical INFANT-catalog screen (389 included) — do not filter the all-age catalog on them.

Files

File	Rows	What it is
sample_metadata_wide.parquet / .csv.gz	154,206	Start here. One row per sample: identifiers, body-site class, every metadata field with <field>__confidence and <field>__route, subject/timepoint, run accessions, cohort, links
sample_determinations.parquet	618,898	Long form: one row per sample × field with evidence_source, evidence_locator, evidence_quote (≤12 words verbatim), route, scope, confidence
study_metadata_wide.parquet / .csv	389	One row per included study (BioProject): title, counts, triage evidence, recoverability tiers, per-field coverage (cov_*), cohort, links
runs.parquet	174,022	Run → sample → study with library/instrument fields (join key into ENA/SRA)
sample_subjects.parquet	154,206	Subject and timepoint resolution per sample (subject_key, role infant/mother/other, t_index)
cohorts.csv	373	Cohort clusters (studies + papers sharing a cohort) with unique-infant estimates
study_paper_links.csv	973	Study ↔ paper links with PMID/PMCID/DOI and link method
sandpiper_flag_table.csv	20	Single source for every Sandpiper flag (field, level, site label, definition, used) — v1.2.2
sandpiper_study_panel_status.csv	389	Per-study panel scope label and n breakdown (age_scope of profiled rows) — v1.2.1
sandpiper_auditor_candidate_runs.csv	375	Runs whose organism label is a non-human host or a named microbe, with the v1.2.1 action taken — v1.2.1
review_genomic_only_samples.csv, review_named_microbe_metagenomic_samples.csv, review_amplicon_runs.csv	3158 / 70 / 39	Owner review lists (library_source GENOMIC on every run; METAGENOMIC runs with a named-microbe organism; AMPLICON-strategy runs) — v1.2.1
universe_studies_all.parquet	9,579	Every screened study with verdict, reason code and evidence
parent_biosamples.parquet	16	Parent BioSamples of the run-level rows (provenance only; excluded from every count)
study_verdict_history.parquet	32,231	Every triage-stage verdict per study (screen, rubric, Sonnet replicates, Opus adjudication, growth waves, BioSample re-judge, human-review rounds, final) with model, confidence, evidence and source artifact
value_history.parquet	45,591	Determinations that are not current: superseded, rejected, dropped by the group-statement audit, duplicates, moved parent rows, auditor findings — each with status, reason, replaced_by
sample_determinations_superseded.parquet	223	Subset of the above kept in the v1.1 layout (run-level supersessions + v1.2 age corrections)
confidence_tiers.csv	29	The engine confidence tiers: route × determiner × scope with the fixed confidence values they emit
tier_field_precision.csv	49	Empirical precision of each route × confidence tier per field against the cMD gold join (hi-res and full sets)
sample_unit_classification_by_study.csv	67	Per-study sample-unit class (A run-keyed, C technical multi-run, X cross-study BioSample)
human_review_queue.csv	52	Studies the pipeline could not decide, with the reason
field_coverage_summary.csv, study_field_coverage_matrix.csv		Coverage per field and per study × field
extraction_gold_eval_hires.csv		Precision/recall vs curatedMetagenomicData
sample_determinations_all.parquet	620,214	Current and retired determinations: sample_determinations columns + release_added, release_retired (null = current), package_added, retired_reason, retired_change_stage — the per-field value timeline — R2026.1
releases.csv	6	Release registry: one row per release id (package version, date, tags, DOI, headline counts, notes file) — R2026.1
RELEASE_NOTES_R2026.1.md		Generated diff report vs the previous package (studies/samples, coverage, verdict flips, findings, Sandpiper, gold, schema) — R2026.1
contribute_worklist.csv	439	Community-contribution worklist: one row per OPEN included/uncertain study (≥ 1 of the six fields age/delivery/feeding/preterm/antibiotics/probiotic below 0.5 coverage on catalog_scope), ranked by priority_score, with blocker_code, unlock_text and a prefilled contribution Issue URL — R2026.2
contribute_worklist_fields.csv	2,634	Study × field detail of the worklist (coverage, best recoverability tier, field-level blocker, evidence) — R2026.2
registry_studies.parquet	54,410	Registry tier: one row per ENA study of the human sh...

Data package data-v1.19.1

Microbiome Repo — data package v1.19.1 (2026-10-10)

One catalog, two tiers, all ages. The registry tables (registry_*) cover every public human shotgun-metagenome study in ENA/SRA/DDBJ — 64,369 studies, all body sites and ages — with archive-only classification (host, UBERON-anchored body sites, life stages, assay, access) and the harvested BioSample tier (637,397 BioSamples with normalised site / age / sex / country). The catalog tables (gut_*) are the curated tier: every registry study of the human gut, all ages — 2,946 studies, 600,643 samples, 4,137,747 sample × field values — with age at collection, sex, BMI, country, health condition (controlled vocabulary), antibiotic exposure, subject and timepoint recovered from archive attributes (R1), supplementary tables (R2), paper full text (R3) and abstracts (R4); every value carries a verbatim evidence quote (≤ 12 words), a labelled source, a route and a confidence. Coverage over all gut samples: age 28%, sex 32%, country 92%, health condition 49%, antibiotic exposure 19%.

Start here. gut_studies.parquet (one row per study: classification, papers, per-field coverage, age-category and health-condition mix, curated depth) → gut_sample_metadata_wide.parquet (one row per sample: every field with __route / __confidence, age_category + basis, body_site_class) → gut_sample_determinations.parquet (the evidence behind every value). registry_studies.parquet is the registry; registry_runs_v1.19.1.parquet (every run) ships as a GitHub Release asset.

Infant extension. The 389 infant-gut studies were curated in greater depth in earlier releases (delivery mode, feeding mode, preterm status, gestational age, birth weight, maternal antibiotics, probiotics, HMO supplementation, NEC). Those rows are present in the gut_* tables (curated_source = infant_catalog, infant_scope = true reproduces the former infant catalog exactly: 72,358 samples) and the original deep tables (sample_metadata_wide, sample_determinations, study_metadata_wide, cohorts, universe_studies_all, sandpiper_*, …) remain in the package for reproducibility; the paragraphs and headline numbers of that extension are kept below under Infant extension (historical description).

Infant extension (historical description)

The registry_* tables cover every public human shotgun-metagenome study in ENA/SRA/DDBJ (all body sites, all ages; archive-only classification plus the harvested BioSample tier). All other tables belong to the first curated scope, the Infant Gut Shotgun-Metagenome Catalog — the description below is that scope's.

A curated, evidence-linked catalog of every public shotgun-metagenome study of the human infant gut (0–36 months) in the INSDC archives (ENA/SRA/DDBJ), with per-sample metadata recovered from archive attributes, supplementary tables and papers.

Headline numbers (computed from the tables in this package; see CHANGELOG v1.2 and v1.2.2). 389 included studies · 154,206 catalog samples (153,685 BioSample units + 521 run units from 6 one-BioSample-per-infant deposits; their 16 parent BioSamples are listed in parent_biosamples.parquet and counted nowhere else) · 174,022 sequencing runs · 618,898 sample × field determinations, each with a verbatim evidence quote (9,559 of them are out-of-scope ages that document adult_age_flag) · age scope: 67,598 samples with per-sample or per-subject infant evidence + 7,097 in studies triage judged all-infant = 74,695 age-scope infant samples, of which 72,358 are also gut/unknown body site = catalog_scope, the headline denominator (2,900 age-scope infant rows are body-site excluded/linked: skin, milk, oral, mother…); 14,410 samples sit in mixed-age deposits and 49,556 in studies without a per-study infant estimate (age unknown); 9,559 carry an out-of-range age (adult_flagged); 5,986 are mother/other samples · age at collection ≤ 1,100 d recovered for 53,669 of 74,695 age-scope infant samples (72%) for 52,649 of 72,358 catalog-scope samples (73%) and for 62,230 of 146,336 body-site infant-scope samples (43%; v1.2.0 said 54,105 of 146,631 before the v1.2.2 host/isolate exclusions; the v1.1 figure 53,802 was computed before the run-unit fix and the site's 54,150 included the 16 parent rows and rows later superseded) · precision vs curatedMetagenomicData (hi-res set, 3,498 gold rows; extraction_gold_eval_hires.csv): age 0.99, delivery mode 0.995, preterm status 1.00 (full set of 3,670 rows: age 0.976, delivery 0.995, preterm 1.00 — see tier_field_precision.csv, column gold_set).

Universe screened: 9,579 candidate studies (all METAGENOMIC WGS/WXS runs in ENA with a human signal, no taxon filter) → 389 included, 52 unresolvable after two full-text review rounds; recall on the 22 curatedMetagenomicData infant BioProjects: 17/22 by the automated cascade, 18/22 after the human-review pass (one study, decision_stage=session_model_review), the rest excluded with a coded reason (universe_studies_all.parquet).

Catalog membership: a study is in the catalog when it is in gut_studies.parquet (samples: gut_sample_metadata_wide.parquet). The catalog_status / triage_verdict / reason_code columns of universe_studies_all.parquet and study_metadata_wide.parquet are the historical INFANT-catalog screen (389 included) — do not filter the all-age catalog on them.

Files

File	Rows	What it is
sample_metadata_wide.parquet / .csv.gz	154,206	Start here. One row per sample: identifiers, body-site class, every metadata field with <field>__confidence and <field>__route, subject/timepoint, run accessions, cohort, links
sample_determinations.parquet	618,898	Long form: one row per sample × field with evidence_source, evidence_locator, evidence_quote (≤12 words verbatim), route, scope, confidence
study_metadata_wide.parquet / .csv	389	One row per included study (BioProject): title, counts, triage evidence, recoverability tiers, per-field coverage (cov_*), cohort, links
runs.parquet	174,022	Run → sample → study with library/instrument fields (join key into ENA/SRA)
sample_subjects.parquet	154,206	Subject and timepoint resolution per sample (subject_key, role infant/mother/other, t_index)
cohorts.csv	373	Cohort clusters (studies + papers sharing a cohort) with unique-infant estimates
study_paper_links.csv	973	Study ↔ paper links with PMID/PMCID/DOI and link method
sandpiper_flag_table.csv	20	Single source for every Sandpiper flag (field, level, site label, definition, used) — v1.2.2
sandpiper_study_panel_status.csv	389	Per-study panel scope label and n breakdown (age_scope of profiled rows) — v1.2.1
sandpiper_auditor_candidate_runs.csv	375	Runs whose organism label is a non-human host or a named microbe, with the v1.2.1 action taken — v1.2.1
review_genomic_only_samples.csv, review_named_microbe_metagenomic_samples.csv, review_amplicon_runs.csv	3158 / 70 / 39	Owner review lists (library_source GENOMIC on every run; METAGENOMIC runs with a named-microbe organism; AMPLICON-strategy runs) — v1.2.1
universe_studies_all.parquet	9,579	Every screened study with verdict, reason code and evidence
parent_biosamples.parquet	16	Parent BioSamples of the run-level rows (provenance only; excluded from every count)
study_verdict_history.parquet	32,231	Every triage-stage verdict per study (screen, rubric, Sonnet replicates, Opus adjudication, growth waves, BioSample re-judge, human-review rounds, final) with model, confidence, evidence and source artifact
value_history.parquet	45,591	Determinations that are not current: superseded, rejected, dropped by the group-statement audit, duplicates, moved parent rows, auditor findings — each with status, reason, replaced_by
sample_determinations_superseded.parquet	223	Subset of the above kept in the v1.1 layout (run-level supersessions + v1.2 age corrections)
confidence_tiers.csv	29	The engine confidence tiers: route × determiner × scope with the fixed confidence values they emit
tier_field_precision.csv	49	Empirical precision of each route × confidence tier per field against the cMD gold join (hi-res and full sets)
sample_unit_classification_by_study.csv	67	Per-study sample-unit class (A run-keyed, C technical multi-run, X cross-study BioSample)
human_review_queue.csv	52	Studies the pipeline could not decide, with the reason
field_coverage_summary.csv, study_field_coverage_matrix.csv		Coverage per field and per study × field
extraction_gold_eval_hires.csv		Precision/recall vs curatedMetagenomicData
sample_determinations_all.parquet	620,214	Current and retired determinations: sample_determinations columns + release_added, release_retired (null = current), package_added, retired_reason, retired_change_stage — the per-field value timeline — R2026.1
releases.csv	6	Release registry: one row per release id (package version, date, tags, DOI, headline counts, notes file) — R2026.1
RELEASE_NOTES_R2026.1.md		Generated diff report vs the previous package (studies/samples, coverage, verdict flips, findings, Sandpiper, gold, schema) — R2026.1
contribute_worklist.csv	439	Community-contribution worklist: one row per OPEN included/uncertain study (≥ 1 of the six fields age/delivery/feeding/preterm/antibiotics/probiotic below 0.5 coverage on catalog_scope), ranked by priority_score, with blocker_code, unlock_text and a prefilled contribution Issue URL — R2026.2
contribute_worklist_fields.csv	2,634	Study × field detail of the worklist (coverage, best recoverability tier, field-level blocker, evidence) — R2026.2
registry_studies.parquet	54,410	Registry tier: one row per ENA study of the human sh...

Data package data-v1.19.0

Microbiome Repo — data package v1.19.0 (2026-10-10)

One catalog, two tiers, all ages. The registry tables (registry_*) cover every public human shotgun-metagenome study in ENA/SRA/DDBJ — 64,369 studies, all body sites and ages — with archive-only classification (host, UBERON-anchored body sites, life stages, assay, access) and the harvested BioSample tier (637,397 BioSamples with normalised site / age / sex / country). The catalog tables (gut_*) are the curated tier: every registry study of the human gut, all ages — 2,946 studies, 600,643 samples, 4,127,431 sample × field values — with age at collection, sex, BMI, country, health condition (controlled vocabulary), antibiotic exposure, subject and timepoint recovered from archive attributes (R1), supplementary tables (R2), paper full text (R3) and abstracts (R4); every value carries a verbatim evidence quote (≤ 12 words), a labelled source, a route and a confidence. Coverage over all gut samples: age 28%, sex 32%, country 92%, health condition 49%, antibiotic exposure 19%.

Start here. gut_studies.parquet (one row per study: classification, papers, per-field coverage, age-category and health-condition mix, curated depth) → gut_sample_metadata_wide.parquet (one row per sample: every field with __route / __confidence, age_category + basis, body_site_class) → gut_sample_determinations.parquet (the evidence behind every value). registry_studies.parquet is the registry; registry_runs_v1.19.0.parquet (every run) ships as a GitHub Release asset.

Infant extension. The 389 infant-gut studies were curated in greater depth in earlier releases (delivery mode, feeding mode, preterm status, gestational age, birth weight, maternal antibiotics, probiotics, HMO supplementation, NEC). Those rows are present in the gut_* tables (curated_source = infant_catalog, infant_scope = true reproduces the former infant catalog exactly: 72,358 samples) and the original deep tables (sample_metadata_wide, sample_determinations, study_metadata_wide, cohorts, universe_studies_all, sandpiper_*, …) remain in the package for reproducibility; the paragraphs and headline numbers of that extension are kept below under Infant extension (historical description).

Infant extension (historical description)

The registry_* tables cover every public human shotgun-metagenome study in ENA/SRA/DDBJ (all body sites, all ages; archive-only classification plus the harvested BioSample tier). All other tables belong to the first curated scope, the Infant Gut Shotgun-Metagenome Catalog — the description below is that scope's.

A curated, evidence-linked catalog of every public shotgun-metagenome study of the human infant gut (0–36 months) in the INSDC archives (ENA/SRA/DDBJ), with per-sample metadata recovered from archive attributes, supplementary tables and papers.

Headline numbers (computed from the tables in this package; see CHANGELOG v1.2 and v1.2.2). 389 included studies · 154,206 catalog samples (153,685 BioSample units + 521 run units from 6 one-BioSample-per-infant deposits; their 16 parent BioSamples are listed in parent_biosamples.parquet and counted nowhere else) · 174,022 sequencing runs · 618,898 sample × field determinations, each with a verbatim evidence quote (9,559 of them are out-of-scope ages that document adult_age_flag) · age scope: 67,598 samples with per-sample or per-subject infant evidence + 7,097 in studies triage judged all-infant = 74,695 age-scope infant samples, of which 72,358 are also gut/unknown body site = catalog_scope, the headline denominator (2,900 age-scope infant rows are body-site excluded/linked: skin, milk, oral, mother…); 14,410 samples sit in mixed-age deposits and 49,556 in studies without a per-study infant estimate (age unknown); 9,559 carry an out-of-range age (adult_flagged); 5,986 are mother/other samples · age at collection ≤ 1,100 d recovered for 53,669 of 74,695 age-scope infant samples (72%) for 52,649 of 72,358 catalog-scope samples (73%) and for 62,230 of 146,336 body-site infant-scope samples (43%; v1.2.0 said 54,105 of 146,631 before the v1.2.2 host/isolate exclusions; the v1.1 figure 53,802 was computed before the run-unit fix and the site's 54,150 included the 16 parent rows and rows later superseded) · precision vs curatedMetagenomicData (hi-res set, 3,498 gold rows; extraction_gold_eval_hires.csv): age 0.99, delivery mode 0.995, preterm status 1.00 (full set of 3,670 rows: age 0.976, delivery 0.995, preterm 1.00 — see tier_field_precision.csv, column gold_set).

Universe screened: 9,579 candidate studies (all METAGENOMIC WGS/WXS runs in ENA with a human signal, no taxon filter) → 389 included, 52 unresolvable after two full-text review rounds; recall on the 22 curatedMetagenomicData infant BioProjects: 17/22 by the automated cascade, 18/22 after the human-review pass (one study, decision_stage=session_model_review), the rest excluded with a coded reason (universe_studies_all.parquet).

Catalog membership: a study is in the catalog when it is in gut_studies.parquet (samples: gut_sample_metadata_wide.parquet). The catalog_status / triage_verdict / reason_code columns of universe_studies_all.parquet and study_metadata_wide.parquet are the historical INFANT-catalog screen (389 included) — do not filter the all-age catalog on them.

Files

File	Rows	What it is
sample_metadata_wide.parquet / .csv.gz	154,206	Start here. One row per sample: identifiers, body-site class, every metadata field with <field>__confidence and <field>__route, subject/timepoint, run accessions, cohort, links
sample_determinations.parquet	618,898	Long form: one row per sample × field with evidence_source, evidence_locator, evidence_quote (≤12 words verbatim), route, scope, confidence
study_metadata_wide.parquet / .csv	389	One row per included study (BioProject): title, counts, triage evidence, recoverability tiers, per-field coverage (cov_*), cohort, links
runs.parquet	174,022	Run → sample → study with library/instrument fields (join key into ENA/SRA)
sample_subjects.parquet	154,206	Subject and timepoint resolution per sample (subject_key, role infant/mother/other, t_index)
cohorts.csv	373	Cohort clusters (studies + papers sharing a cohort) with unique-infant estimates
study_paper_links.csv	973	Study ↔ paper links with PMID/PMCID/DOI and link method
sandpiper_flag_table.csv	20	Single source for every Sandpiper flag (field, level, site label, definition, used) — v1.2.2
sandpiper_study_panel_status.csv	389	Per-study panel scope label and n breakdown (age_scope of profiled rows) — v1.2.1
sandpiper_auditor_candidate_runs.csv	375	Runs whose organism label is a non-human host or a named microbe, with the v1.2.1 action taken — v1.2.1
review_genomic_only_samples.csv, review_named_microbe_metagenomic_samples.csv, review_amplicon_runs.csv	3158 / 70 / 39	Owner review lists (library_source GENOMIC on every run; METAGENOMIC runs with a named-microbe organism; AMPLICON-strategy runs) — v1.2.1
universe_studies_all.parquet	9,579	Every screened study with verdict, reason code and evidence
parent_biosamples.parquet	16	Parent BioSamples of the run-level rows (provenance only; excluded from every count)
study_verdict_history.parquet	32,231	Every triage-stage verdict per study (screen, rubric, Sonnet replicates, Opus adjudication, growth waves, BioSample re-judge, human-review rounds, final) with model, confidence, evidence and source artifact
value_history.parquet	45,591	Determinations that are not current: superseded, rejected, dropped by the group-statement audit, duplicates, moved parent rows, auditor findings — each with status, reason, replaced_by
sample_determinations_superseded.parquet	223	Subset of the above kept in the v1.1 layout (run-level supersessions + v1.2 age corrections)
confidence_tiers.csv	29	The engine confidence tiers: route × determiner × scope with the fixed confidence values they emit
tier_field_precision.csv	49	Empirical precision of each route × confidence tier per field against the cMD gold join (hi-res and full sets)
sample_unit_classification_by_study.csv	67	Per-study sample-unit class (A run-keyed, C technical multi-run, X cross-study BioSample)
human_review_queue.csv	52	Studies the pipeline could not decide, with the reason
field_coverage_summary.csv, study_field_coverage_matrix.csv		Coverage per field and per study × field
extraction_gold_eval_hires.csv		Precision/recall vs curatedMetagenomicData
sample_determinations_all.parquet	620,214	Current and retired determinations: sample_determinations columns + release_added, release_retired (null = current), package_added, retired_reason, retired_change_stage — the per-field value timeline — R2026.1
releases.csv	6	Release registry: one row per release id (package version, date, tags, DOI, headline counts, notes file) — R2026.1
RELEASE_NOTES_R2026.1.md		Generated diff report vs the previous package (studies/samples, coverage, verdict flips, findings, Sandpiper, gold, schema) — R2026.1
contribute_worklist.csv	439	Community-contribution worklist: one row per OPEN included/uncertain study (≥ 1 of the six fields age/delivery/feeding/preterm/antibiotics/probiotic below 0.5 coverage on catalog_scope), ranked by priority_score, with blocker_code, unlock_text and a prefilled contribution Issue URL — R2026.2
contribute_worklist_fields.csv	2,634	Study × field detail of the worklist (coverage, best recoverability tier, field-level blocker, evidence) — R2026.2
registry_studies.parquet	54,410	Registry tier: one row per ENA study of the human sh...

Data package data-v1.18.2

Microbiome Repo — data package v1.18.2 (2026-10-09)

One catalog, two tiers, all ages. The registry tables (registry_*) cover every public human shotgun-metagenome study in ENA/SRA/DDBJ — 64,369 studies, all body sites and ages — with archive-only classification (host, UBERON-anchored body sites, life stages, assay, access) and the harvested BioSample tier (637,397 BioSamples with normalised site / age / sex / country). The catalog tables (gut_*) are the curated tier: every registry study of the human gut, all ages — 2,946 studies, 600,643 samples, 3,920,362 sample × field values — with age at collection, sex, BMI, country, health condition (controlled vocabulary), antibiotic exposure, subject and timepoint recovered from archive attributes (R1), supplementary tables (R2), paper full text (R3) and abstracts (R4); every value carries a verbatim evidence quote (≤ 12 words), a labelled source, a route and a confidence. Coverage over all gut samples: age 28%, sex 32%, country 92%, health condition 49%, antibiotic exposure 19%.

Start here. gut_studies.parquet (one row per study: classification, papers, per-field coverage, age-category and health-condition mix, curated depth) → gut_sample_metadata_wide.parquet (one row per sample: every field with __route / __confidence, age_category + basis, body_site_class) → gut_sample_determinations.parquet (the evidence behind every value). registry_studies.parquet is the registry; registry_runs_v1.18.2.parquet (every run) ships as a GitHub Release asset.

Infant extension. The 389 infant-gut studies were curated in greater depth in earlier releases (delivery mode, feeding mode, preterm status, gestational age, birth weight, maternal antibiotics, probiotics, HMO supplementation, NEC). Those rows are present in the gut_* tables (curated_source = infant_catalog, infant_scope = true reproduces the former infant catalog exactly: 72,358 samples) and the original deep tables (sample_metadata_wide, sample_determinations, study_metadata_wide, cohorts, universe_studies_all, sandpiper_*, …) remain in the package for reproducibility; the paragraphs and headline numbers of that extension are kept below under Infant extension (historical description).

Infant extension (historical description)

The registry_* tables cover every public human shotgun-metagenome study in ENA/SRA/DDBJ (all body sites, all ages; archive-only classification plus the harvested BioSample tier). All other tables belong to the first curated scope, the Infant Gut Shotgun-Metagenome Catalog — the description below is that scope's.

A curated, evidence-linked catalog of every public shotgun-metagenome study of the human infant gut (0–36 months) in the INSDC archives (ENA/SRA/DDBJ), with per-sample metadata recovered from archive attributes, supplementary tables and papers.

Headline numbers (computed from the tables in this package; see CHANGELOG v1.2 and v1.2.2). 389 included studies · 154,206 catalog samples (153,685 BioSample units + 521 run units from 6 one-BioSample-per-infant deposits; their 16 parent BioSamples are listed in parent_biosamples.parquet and counted nowhere else) · 174,022 sequencing runs · 618,898 sample × field determinations, each with a verbatim evidence quote (9,559 of them are out-of-scope ages that document adult_age_flag) · age scope: 67,598 samples with per-sample or per-subject infant evidence + 7,097 in studies triage judged all-infant = 74,695 age-scope infant samples, of which 72,358 are also gut/unknown body site = catalog_scope, the headline denominator (2,900 age-scope infant rows are body-site excluded/linked: skin, milk, oral, mother…); 14,410 samples sit in mixed-age deposits and 49,556 in studies without a per-study infant estimate (age unknown); 9,559 carry an out-of-range age (adult_flagged); 5,986 are mother/other samples · age at collection ≤ 1,100 d recovered for 53,669 of 74,695 age-scope infant samples (72%) for 52,649 of 72,358 catalog-scope samples (73%) and for 62,230 of 146,336 body-site infant-scope samples (43%; v1.2.0 said 54,105 of 146,631 before the v1.2.2 host/isolate exclusions; the v1.1 figure 53,802 was computed before the run-unit fix and the site's 54,150 included the 16 parent rows and rows later superseded) · precision vs curatedMetagenomicData (hi-res set, 3,498 gold rows; extraction_gold_eval_hires.csv): age 0.99, delivery mode 0.995, preterm status 1.00 (full set of 3,670 rows: age 0.976, delivery 0.995, preterm 1.00 — see tier_field_precision.csv, column gold_set).

Universe screened: 9,579 candidate studies (all METAGENOMIC WGS/WXS runs in ENA with a human signal, no taxon filter) → 389 included, 52 unresolvable after two full-text review rounds; recall on the 22 curatedMetagenomicData infant BioProjects: 17/22 by the automated cascade, 18/22 after the human-review pass (one study, decision_stage=session_model_review), the rest excluded with a coded reason (universe_studies_all.parquet).

Catalog membership: a study is in the catalog when it is in gut_studies.parquet (samples: gut_sample_metadata_wide.parquet). The catalog_status / triage_verdict / reason_code columns of universe_studies_all.parquet and study_metadata_wide.parquet are the historical INFANT-catalog screen (389 included) — do not filter the all-age catalog on them.

Files

File	Rows	What it is
sample_metadata_wide.parquet / .csv.gz	154,206	Start here. One row per sample: identifiers, body-site class, every metadata field with <field>__confidence and <field>__route, subject/timepoint, run accessions, cohort, links
sample_determinations.parquet	618,898	Long form: one row per sample × field with evidence_source, evidence_locator, evidence_quote (≤12 words verbatim), route, scope, confidence
study_metadata_wide.parquet / .csv	389	One row per included study (BioProject): title, counts, triage evidence, recoverability tiers, per-field coverage (cov_*), cohort, links
runs.parquet	174,022	Run → sample → study with library/instrument fields (join key into ENA/SRA)
sample_subjects.parquet	154,206	Subject and timepoint resolution per sample (subject_key, role infant/mother/other, t_index)
cohorts.csv	373	Cohort clusters (studies + papers sharing a cohort) with unique-infant estimates
study_paper_links.csv	973	Study ↔ paper links with PMID/PMCID/DOI and link method
sandpiper_flag_table.csv	20	Single source for every Sandpiper flag (field, level, site label, definition, used) — v1.2.2
sandpiper_study_panel_status.csv	389	Per-study panel scope label and n breakdown (age_scope of profiled rows) — v1.2.1
sandpiper_auditor_candidate_runs.csv	375	Runs whose organism label is a non-human host or a named microbe, with the v1.2.1 action taken — v1.2.1
review_genomic_only_samples.csv, review_named_microbe_metagenomic_samples.csv, review_amplicon_runs.csv	3158 / 70 / 39	Owner review lists (library_source GENOMIC on every run; METAGENOMIC runs with a named-microbe organism; AMPLICON-strategy runs) — v1.2.1
universe_studies_all.parquet	9,579	Every screened study with verdict, reason code and evidence
parent_biosamples.parquet	16	Parent BioSamples of the run-level rows (provenance only; excluded from every count)
study_verdict_history.parquet	32,231	Every triage-stage verdict per study (screen, rubric, Sonnet replicates, Opus adjudication, growth waves, BioSample re-judge, human-review rounds, final) with model, confidence, evidence and source artifact
value_history.parquet	45,591	Determinations that are not current: superseded, rejected, dropped by the group-statement audit, duplicates, moved parent rows, auditor findings — each with status, reason, replaced_by
sample_determinations_superseded.parquet	223	Subset of the above kept in the v1.1 layout (run-level supersessions + v1.2 age corrections)
confidence_tiers.csv	29	The engine confidence tiers: route × determiner × scope with the fixed confidence values they emit
tier_field_precision.csv	49	Empirical precision of each route × confidence tier per field against the cMD gold join (hi-res and full sets)
sample_unit_classification_by_study.csv	67	Per-study sample-unit class (A run-keyed, C technical multi-run, X cross-study BioSample)
human_review_queue.csv	52	Studies the pipeline could not decide, with the reason
field_coverage_summary.csv, study_field_coverage_matrix.csv		Coverage per field and per study × field
extraction_gold_eval_hires.csv		Precision/recall vs curatedMetagenomicData
sample_determinations_all.parquet	620,214	Current and retired determinations: sample_determinations columns + release_added, release_retired (null = current), package_added, retired_reason, retired_change_stage — the per-field value timeline — R2026.1
releases.csv	6	Release registry: one row per release id (package version, date, tags, DOI, headline counts, notes file) — R2026.1
RELEASE_NOTES_R2026.1.md		Generated diff report vs the previous package (studies/samples, coverage, verdict flips, findings, Sandpiper, gold, schema) — R2026.1
contribute_worklist.csv	439	Community-contribution worklist: one row per OPEN included/uncertain study (≥ 1 of the six fields age/delivery/feeding/preterm/antibiotics/probiotic below 0.5 coverage on catalog_scope), ranked by priority_score, with blocker_code, unlock_text and a prefilled contribution Issue URL — R2026.2
contribute_worklist_fields.csv	2,634	Study × field detail of the worklist (coverage, best recoverability tier, field-level blocker, evidence) — R2026.2
registry_studies.parquet	54,410	Registry tier: one row per ENA study of the human sh...

Data package data-v1.18.1

Microbiome Repo — data package v1.18.1 (2026-10-09)

One catalog, two tiers, all ages. The registry tables (registry_*) cover every public human shotgun-metagenome study in ENA/SRA/DDBJ — 64,369 studies, all body sites and ages — with archive-only classification (host, UBERON-anchored body sites, life stages, assay, access) and the harvested BioSample tier (637,397 BioSamples with normalised site / age / sex / country). The catalog tables (gut_*) are the curated tier: every registry study of the human gut, all ages — 2,946 studies, 600,643 samples, 3,920,362 sample × field values — with age at collection, sex, BMI, country, health condition (controlled vocabulary), antibiotic exposure, subject and timepoint recovered from archive attributes (R1), supplementary tables (R2), paper full text (R3) and abstracts (R4); every value carries a verbatim evidence quote (≤ 12 words), a labelled source, a route and a confidence. Coverage over all gut samples: age 28%, sex 32%, country 92%, health condition 49%, antibiotic exposure 19%.

Start here. gut_studies.parquet (one row per study: classification, papers, per-field coverage, age-category and health-condition mix, curated depth) → gut_sample_metadata_wide.parquet (one row per sample: every field with __route / __confidence, age_category + basis, body_site_class) → gut_sample_determinations.parquet (the evidence behind every value). registry_studies.parquet is the registry; registry_runs_v1.18.1.parquet (every run) ships as a GitHub Release asset.

Infant extension. The 389 infant-gut studies were curated in greater depth in earlier releases (delivery mode, feeding mode, preterm status, gestational age, birth weight, maternal antibiotics, probiotics, HMO supplementation, NEC). Those rows are present in the gut_* tables (curated_source = infant_catalog, infant_scope = true reproduces the former infant catalog exactly: 72,358 samples) and the original deep tables (sample_metadata_wide, sample_determinations, study_metadata_wide, cohorts, universe_studies_all, sandpiper_*, …) remain in the package for reproducibility; the paragraphs and headline numbers of that extension are kept below under Infant extension (historical description).

Infant extension (historical description)

The registry_* tables cover every public human shotgun-metagenome study in ENA/SRA/DDBJ (all body sites, all ages; archive-only classification plus the harvested BioSample tier). All other tables belong to the first curated scope, the Infant Gut Shotgun-Metagenome Catalog — the description below is that scope's.

A curated, evidence-linked catalog of every public shotgun-metagenome study of the human infant gut (0–36 months) in the INSDC archives (ENA/SRA/DDBJ), with per-sample metadata recovered from archive attributes, supplementary tables and papers.

Headline numbers (computed from the tables in this package; see CHANGELOG v1.2 and v1.2.2). 389 included studies · 154,206 catalog samples (153,685 BioSample units + 521 run units from 6 one-BioSample-per-infant deposits; their 16 parent BioSamples are listed in parent_biosamples.parquet and counted nowhere else) · 174,022 sequencing runs · 618,898 sample × field determinations, each with a verbatim evidence quote (9,559 of them are out-of-scope ages that document adult_age_flag) · age scope: 67,598 samples with per-sample or per-subject infant evidence + 7,097 in studies triage judged all-infant = 74,695 age-scope infant samples, of which 72,358 are also gut/unknown body site = catalog_scope, the headline denominator (2,900 age-scope infant rows are body-site excluded/linked: skin, milk, oral, mother…); 14,410 samples sit in mixed-age deposits and 49,556 in studies without a per-study infant estimate (age unknown); 9,559 carry an out-of-range age (adult_flagged); 5,986 are mother/other samples · age at collection ≤ 1,100 d recovered for 53,669 of 74,695 age-scope infant samples (72%) for 52,649 of 72,358 catalog-scope samples (73%) and for 62,230 of 146,336 body-site infant-scope samples (43%; v1.2.0 said 54,105 of 146,631 before the v1.2.2 host/isolate exclusions; the v1.1 figure 53,802 was computed before the run-unit fix and the site's 54,150 included the 16 parent rows and rows later superseded) · precision vs curatedMetagenomicData (hi-res set, 3,498 gold rows; extraction_gold_eval_hires.csv): age 0.99, delivery mode 0.995, preterm status 1.00 (full set of 3,670 rows: age 0.976, delivery 0.995, preterm 1.00 — see tier_field_precision.csv, column gold_set).

Universe screened: 9,579 candidate studies (all METAGENOMIC WGS/WXS runs in ENA with a human signal, no taxon filter) → 389 included, 52 unresolvable after two full-text review rounds; recall on the 22 curatedMetagenomicData infant BioProjects: 17/22 by the automated cascade, 18/22 after the human-review pass (one study, decision_stage=session_model_review), the rest excluded with a coded reason (universe_studies_all.parquet).

Catalog membership: a study is in the catalog when it is in gut_studies.parquet (samples: gut_sample_metadata_wide.parquet). The catalog_status / triage_verdict / reason_code columns of universe_studies_all.parquet and study_metadata_wide.parquet are the historical INFANT-catalog screen (389 included) — do not filter the all-age catalog on them.

Files

File	Rows	What it is
sample_metadata_wide.parquet / .csv.gz	154,206	Start here. One row per sample: identifiers, body-site class, every metadata field with <field>__confidence and <field>__route, subject/timepoint, run accessions, cohort, links
sample_determinations.parquet	618,898	Long form: one row per sample × field with evidence_source, evidence_locator, evidence_quote (≤12 words verbatim), route, scope, confidence
study_metadata_wide.parquet / .csv	389	One row per included study (BioProject): title, counts, triage evidence, recoverability tiers, per-field coverage (cov_*), cohort, links
runs.parquet	174,022	Run → sample → study with library/instrument fields (join key into ENA/SRA)
sample_subjects.parquet	154,206	Subject and timepoint resolution per sample (subject_key, role infant/mother/other, t_index)
cohorts.csv	373	Cohort clusters (studies + papers sharing a cohort) with unique-infant estimates
study_paper_links.csv	973	Study ↔ paper links with PMID/PMCID/DOI and link method
sandpiper_flag_table.csv	20	Single source for every Sandpiper flag (field, level, site label, definition, used) — v1.2.2
sandpiper_study_panel_status.csv	389	Per-study panel scope label and n breakdown (age_scope of profiled rows) — v1.2.1
sandpiper_auditor_candidate_runs.csv	375	Runs whose organism label is a non-human host or a named microbe, with the v1.2.1 action taken — v1.2.1
review_genomic_only_samples.csv, review_named_microbe_metagenomic_samples.csv, review_amplicon_runs.csv	3158 / 70 / 39	Owner review lists (library_source GENOMIC on every run; METAGENOMIC runs with a named-microbe organism; AMPLICON-strategy runs) — v1.2.1
universe_studies_all.parquet	9,579	Every screened study with verdict, reason code and evidence
parent_biosamples.parquet	16	Parent BioSamples of the run-level rows (provenance only; excluded from every count)
study_verdict_history.parquet	32,231	Every triage-stage verdict per study (screen, rubric, Sonnet replicates, Opus adjudication, growth waves, BioSample re-judge, human-review rounds, final) with model, confidence, evidence and source artifact
value_history.parquet	45,591	Determinations that are not current: superseded, rejected, dropped by the group-statement audit, duplicates, moved parent rows, auditor findings — each with status, reason, replaced_by
sample_determinations_superseded.parquet	223	Subset of the above kept in the v1.1 layout (run-level supersessions + v1.2 age corrections)
confidence_tiers.csv	29	The engine confidence tiers: route × determiner × scope with the fixed confidence values they emit
tier_field_precision.csv	49	Empirical precision of each route × confidence tier per field against the cMD gold join (hi-res and full sets)
sample_unit_classification_by_study.csv	67	Per-study sample-unit class (A run-keyed, C technical multi-run, X cross-study BioSample)
human_review_queue.csv	52	Studies the pipeline could not decide, with the reason
field_coverage_summary.csv, study_field_coverage_matrix.csv		Coverage per field and per study × field
extraction_gold_eval_hires.csv		Precision/recall vs curatedMetagenomicData
sample_determinations_all.parquet	620,214	Current and retired determinations: sample_determinations columns + release_added, release_retired (null = current), package_added, retired_reason, retired_change_stage — the per-field value timeline — R2026.1
releases.csv	6	Release registry: one row per release id (package version, date, tags, DOI, headline counts, notes file) — R2026.1
RELEASE_NOTES_R2026.1.md		Generated diff report vs the previous package (studies/samples, coverage, verdict flips, findings, Sandpiper, gold, schema) — R2026.1
contribute_worklist.csv	439	Community-contribution worklist: one row per OPEN included/uncertain study (≥ 1 of the six fields age/delivery/feeding/preterm/antibiotics/probiotic below 0.5 coverage on catalog_scope), ranked by priority_score, with blocker_code, unlock_text and a prefilled contribution Issue URL — R2026.2
contribute_worklist_fields.csv	2,634	Study × field detail of the worklist (coverage, best recoverability tier, field-level blocker, evidence) — R2026.2
registry_studies.parquet	54,410	Registry tier: one row per ENA study of the human sh...

Data package data-v1.18.0

Microbiome Repo — data package v1.18.0 (2026-10-09)

One catalog, two tiers, all ages. The registry tables (registry_*) cover every public human shotgun-metagenome study in ENA/SRA/DDBJ — 64,369 studies, all body sites and ages — with archive-only classification (host, UBERON-anchored body sites, life stages, assay, access) and the harvested BioSample tier (634,138 BioSamples with normalised site / age / sex / country). The catalog tables (gut_*) are the curated tier: every registry study of the human gut, all ages — 2,946 studies, 599,230 samples, 3,849,193 sample × field values — with age at collection, sex, BMI, country, health condition (controlled vocabulary), antibiotic exposure, subject and timepoint recovered from archive attributes (R1), supplementary tables (R2), paper full text (R3) and abstracts (R4); every value carries a verbatim evidence quote (≤ 12 words), a labelled source, a route and a confidence. Coverage over all gut samples: age 28%, sex 32%, country 91%, health condition 47%, antibiotic exposure 19%.

Start here. gut_studies.parquet (one row per study: classification, papers, per-field coverage, age-category and health-condition mix, curated depth) → gut_sample_metadata_wide.parquet (one row per sample: every field with __route / __confidence, age_category + basis, body_site_class) → gut_sample_determinations.parquet (the evidence behind every value). registry_studies.parquet is the registry; registry_runs_v1.18.0.parquet (every run) ships as a GitHub Release asset.

Infant extension. The 389 infant-gut studies were curated in greater depth in earlier releases (delivery mode, feeding mode, preterm status, gestational age, birth weight, maternal antibiotics, probiotics, HMO supplementation, NEC). Those rows are present in the gut_* tables (curated_source = infant_catalog, infant_scope = true reproduces the former infant catalog exactly: 72,358 samples) and the original deep tables (sample_metadata_wide, sample_determinations, study_metadata_wide, cohorts, universe_studies_all, sandpiper_*, …) remain in the package for reproducibility; the paragraphs and headline numbers of that extension are kept below under Infant extension (historical description).

Infant extension (historical description)

The registry_* tables cover every public human shotgun-metagenome study in ENA/SRA/DDBJ (all body sites, all ages; archive-only classification plus the harvested BioSample tier). All other tables belong to the first curated scope, the Infant Gut Shotgun-Metagenome Catalog — the description below is that scope's.

A curated, evidence-linked catalog of every public shotgun-metagenome study of the human infant gut (0–36 months) in the INSDC archives (ENA/SRA/DDBJ), with per-sample metadata recovered from archive attributes, supplementary tables and papers.

Headline numbers (computed from the tables in this package; see CHANGELOG v1.2 and v1.2.2). 389 included studies · 154,206 catalog samples (153,685 BioSample units + 521 run units from 6 one-BioSample-per-infant deposits; their 16 parent BioSamples are listed in parent_biosamples.parquet and counted nowhere else) · 174,022 sequencing runs · 618,898 sample × field determinations, each with a verbatim evidence quote (9,559 of them are out-of-scope ages that document adult_age_flag) · age scope: 67,598 samples with per-sample or per-subject infant evidence + 7,097 in studies triage judged all-infant = 74,695 age-scope infant samples, of which 72,358 are also gut/unknown body site = catalog_scope, the headline denominator (2,900 age-scope infant rows are body-site excluded/linked: skin, milk, oral, mother…); 14,410 samples sit in mixed-age deposits and 49,556 in studies without a per-study infant estimate (age unknown); 9,559 carry an out-of-range age (adult_flagged); 5,986 are mother/other samples · age at collection ≤ 1,100 d recovered for 53,669 of 74,695 age-scope infant samples (72%) for 52,649 of 72,358 catalog-scope samples (73%) and for 62,230 of 146,336 body-site infant-scope samples (43%; v1.2.0 said 54,105 of 146,631 before the v1.2.2 host/isolate exclusions; the v1.1 figure 53,802 was computed before the run-unit fix and the site's 54,150 included the 16 parent rows and rows later superseded) · precision vs curatedMetagenomicData (hi-res set, 3,498 gold rows; extraction_gold_eval_hires.csv): age 0.99, delivery mode 0.995, preterm status 1.00 (full set of 3,670 rows: age 0.976, delivery 0.995, preterm 1.00 — see tier_field_precision.csv, column gold_set).

Universe screened: 9,579 candidate studies (all METAGENOMIC WGS/WXS runs in ENA with a human signal, no taxon filter) → 389 included, 52 unresolvable after two full-text review rounds; recall on the 22 curatedMetagenomicData infant BioProjects: 17/22 by the automated cascade, 18/22 after the human-review pass (one study, decision_stage=session_model_review), the rest excluded with a coded reason (universe_studies_all.parquet).

Catalog membership: a study is in the catalog when it is in gut_studies.parquet (samples: gut_sample_metadata_wide.parquet). The catalog_status / triage_verdict / reason_code columns of universe_studies_all.parquet and study_metadata_wide.parquet are the historical INFANT-catalog screen (389 included) — do not filter the all-age catalog on them.

Files

File	Rows	What it is
sample_metadata_wide.parquet / .csv.gz	154,206	Start here. One row per sample: identifiers, body-site class, every metadata field with <field>__confidence and <field>__route, subject/timepoint, run accessions, cohort, links
sample_determinations.parquet	618,898	Long form: one row per sample × field with evidence_source, evidence_locator, evidence_quote (≤12 words verbatim), route, scope, confidence
study_metadata_wide.parquet / .csv	389	One row per included study (BioProject): title, counts, triage evidence, recoverability tiers, per-field coverage (cov_*), cohort, links
runs.parquet	174,022	Run → sample → study with library/instrument fields (join key into ENA/SRA)
sample_subjects.parquet	154,206	Subject and timepoint resolution per sample (subject_key, role infant/mother/other, t_index)
cohorts.csv	373	Cohort clusters (studies + papers sharing a cohort) with unique-infant estimates
study_paper_links.csv	973	Study ↔ paper links with PMID/PMCID/DOI and link method
sandpiper_flag_table.csv	20	Single source for every Sandpiper flag (field, level, site label, definition, used) — v1.2.2
sandpiper_study_panel_status.csv	389	Per-study panel scope label and n breakdown (age_scope of profiled rows) — v1.2.1
sandpiper_auditor_candidate_runs.csv	375	Runs whose organism label is a non-human host or a named microbe, with the v1.2.1 action taken — v1.2.1
review_genomic_only_samples.csv, review_named_microbe_metagenomic_samples.csv, review_amplicon_runs.csv	3158 / 70 / 39	Owner review lists (library_source GENOMIC on every run; METAGENOMIC runs with a named-microbe organism; AMPLICON-strategy runs) — v1.2.1
universe_studies_all.parquet	9,579	Every screened study with verdict, reason code and evidence
parent_biosamples.parquet	16	Parent BioSamples of the run-level rows (provenance only; excluded from every count)
study_verdict_history.parquet	32,231	Every triage-stage verdict per study (screen, rubric, Sonnet replicates, Opus adjudication, growth waves, BioSample re-judge, human-review rounds, final) with model, confidence, evidence and source artifact
value_history.parquet	45,591	Determinations that are not current: superseded, rejected, dropped by the group-statement audit, duplicates, moved parent rows, auditor findings — each with status, reason, replaced_by
sample_determinations_superseded.parquet	223	Subset of the above kept in the v1.1 layout (run-level supersessions + v1.2 age corrections)
confidence_tiers.csv	29	The engine confidence tiers: route × determiner × scope with the fixed confidence values they emit
tier_field_precision.csv	49	Empirical precision of each route × confidence tier per field against the cMD gold join (hi-res and full sets)
sample_unit_classification_by_study.csv	67	Per-study sample-unit class (A run-keyed, C technical multi-run, X cross-study BioSample)
human_review_queue.csv	52	Studies the pipeline could not decide, with the reason
field_coverage_summary.csv, study_field_coverage_matrix.csv		Coverage per field and per study × field
extraction_gold_eval_hires.csv		Precision/recall vs curatedMetagenomicData
sample_determinations_all.parquet	620,214	Current and retired determinations: sample_determinations columns + release_added, release_retired (null = current), package_added, retired_reason, retired_change_stage — the per-field value timeline — R2026.1
releases.csv	6	Release registry: one row per release id (package version, date, tags, DOI, headline counts, notes file) — R2026.1
RELEASE_NOTES_R2026.1.md		Generated diff report vs the previous package (studies/samples, coverage, verdict flips, findings, Sandpiper, gold, schema) — R2026.1
contribute_worklist.csv	439	Community-contribution worklist: one row per OPEN included/uncertain study (≥ 1 of the six fields age/delivery/feeding/preterm/antibiotics/probiotic below 0.5 coverage on catalog_scope), ranked by priority_score, with blocker_code, unlock_text and a prefilled contribution Issue URL — R2026.2
contribute_worklist_fields.csv	2,634	Study × field detail of the worklist (coverage, best recoverability tier, field-level blocker, evidence) — R2026.2
registry_studies.parquet	54,410	Registry tier: one row per ENA study of the human sh...

Data package data-v1.17.1

Microbiome Repo — data package v1.17.1 (2026-10-09)

One catalog, two tiers, all ages. The registry tables (registry_*) cover every public human shotgun-metagenome study in ENA/SRA/DDBJ — 64,369 studies, all body sites and ages — with archive-only classification (host, UBERON-anchored body sites, life stages, assay, access) and the harvested BioSample tier (615,944 BioSamples with normalised site / age / sex / country). The catalog tables (gut_*) are the curated tier: every registry study of the human gut, all ages — 2,893 studies, 583,315 samples, 3,825,853 sample × field values — with age at collection, sex, BMI, country, health condition (controlled vocabulary), antibiotic exposure, subject and timepoint recovered from archive attributes (R1), supplementary tables (R2), paper full text (R3) and abstracts (R4); every value carries a verbatim evidence quote (≤ 12 words), a labelled source, a route and a confidence. Coverage over all gut samples: age 28%, sex 32%, country 92%, health condition 48%, antibiotic exposure 19%.

Start here. gut_studies.parquet (one row per study: classification, papers, per-field coverage, age-category and health-condition mix, curated depth) → gut_sample_metadata_wide.parquet (one row per sample: every field with __route / __confidence, age_category + basis, body_site_class) → gut_sample_determinations.parquet (the evidence behind every value). registry_studies.parquet is the registry; registry_runs_v1.17.1.parquet (every run) ships as a GitHub Release asset.

Infant extension. The 389 infant-gut studies were curated in greater depth in earlier releases (delivery mode, feeding mode, preterm status, gestational age, birth weight, maternal antibiotics, probiotics, HMO supplementation, NEC). Those rows are present in the gut_* tables (curated_source = infant_catalog, infant_scope = true reproduces the former infant catalog exactly: 72,358 samples) and the original deep tables (sample_metadata_wide, sample_determinations, study_metadata_wide, cohorts, universe_studies_all, sandpiper_*, …) remain in the package for reproducibility; the paragraphs and headline numbers of that extension are kept below under Infant extension (historical description).

Infant extension (historical description)

The registry_* tables cover every public human shotgun-metagenome study in ENA/SRA/DDBJ (all body sites, all ages; archive-only classification plus the harvested BioSample tier). All other tables belong to the first curated scope, the Infant Gut Shotgun-Metagenome Catalog — the description below is that scope's.

A curated, evidence-linked catalog of every public shotgun-metagenome study of the human infant gut (0–36 months) in the INSDC archives (ENA/SRA/DDBJ), with per-sample metadata recovered from archive attributes, supplementary tables and papers.

Headline numbers (computed from the tables in this package; see CHANGELOG v1.2 and v1.2.2). 389 included studies · 154,206 catalog samples (153,685 BioSample units + 521 run units from 6 one-BioSample-per-infant deposits; their 16 parent BioSamples are listed in parent_biosamples.parquet and counted nowhere else) · 174,022 sequencing runs · 618,898 sample × field determinations, each with a verbatim evidence quote (9,559 of them are out-of-scope ages that document adult_age_flag) · age scope: 67,598 samples with per-sample or per-subject infant evidence + 7,097 in studies triage judged all-infant = 74,695 age-scope infant samples, of which 72,358 are also gut/unknown body site = catalog_scope, the headline denominator (2,900 age-scope infant rows are body-site excluded/linked: skin, milk, oral, mother…); 14,410 samples sit in mixed-age deposits and 49,556 in studies without a per-study infant estimate (age unknown); 9,559 carry an out-of-range age (adult_flagged); 5,986 are mother/other samples · age at collection ≤ 1,100 d recovered for 53,669 of 74,695 age-scope infant samples (72%) for 52,649 of 72,358 catalog-scope samples (73%) and for 62,230 of 146,336 body-site infant-scope samples (43%; v1.2.0 said 54,105 of 146,631 before the v1.2.2 host/isolate exclusions; the v1.1 figure 53,802 was computed before the run-unit fix and the site's 54,150 included the 16 parent rows and rows later superseded) · precision vs curatedMetagenomicData (hi-res set, 3,498 gold rows; extraction_gold_eval_hires.csv): age 0.99, delivery mode 0.995, preterm status 1.00 (full set of 3,670 rows: age 0.976, delivery 0.995, preterm 1.00 — see tier_field_precision.csv, column gold_set).

Universe screened: 9,579 candidate studies (all METAGENOMIC WGS/WXS runs in ENA with a human signal, no taxon filter) → 389 included, 52 unresolvable after two full-text review rounds; recall on the 22 curatedMetagenomicData infant BioProjects: 17/22 by the automated cascade, 18/22 after the human-review pass (one study, decision_stage=session_model_review), the rest excluded with a coded reason (universe_studies_all.parquet).

Catalog membership: a study is in the catalog when it is in gut_studies.parquet (samples: gut_sample_metadata_wide.parquet). The catalog_status / triage_verdict / reason_code columns of universe_studies_all.parquet and study_metadata_wide.parquet are the historical INFANT-catalog screen (389 included) — do not filter the all-age catalog on them.

Files

File	Rows	What it is
sample_metadata_wide.parquet / .csv.gz	154,206	Start here. One row per sample: identifiers, body-site class, every metadata field with <field>__confidence and <field>__route, subject/timepoint, run accessions, cohort, links
sample_determinations.parquet	618,898	Long form: one row per sample × field with evidence_source, evidence_locator, evidence_quote (≤12 words verbatim), route, scope, confidence
study_metadata_wide.parquet / .csv	389	One row per included study (BioProject): title, counts, triage evidence, recoverability tiers, per-field coverage (cov_*), cohort, links
runs.parquet	174,022	Run → sample → study with library/instrument fields (join key into ENA/SRA)
sample_subjects.parquet	154,206	Subject and timepoint resolution per sample (subject_key, role infant/mother/other, t_index)
cohorts.csv	373	Cohort clusters (studies + papers sharing a cohort) with unique-infant estimates
study_paper_links.csv	973	Study ↔ paper links with PMID/PMCID/DOI and link method
sandpiper_flag_table.csv	20	Single source for every Sandpiper flag (field, level, site label, definition, used) — v1.2.2
sandpiper_study_panel_status.csv	389	Per-study panel scope label and n breakdown (age_scope of profiled rows) — v1.2.1
sandpiper_auditor_candidate_runs.csv	375	Runs whose organism label is a non-human host or a named microbe, with the v1.2.1 action taken — v1.2.1
review_genomic_only_samples.csv, review_named_microbe_metagenomic_samples.csv, review_amplicon_runs.csv	3158 / 70 / 39	Owner review lists (library_source GENOMIC on every run; METAGENOMIC runs with a named-microbe organism; AMPLICON-strategy runs) — v1.2.1
universe_studies_all.parquet	9,579	Every screened study with verdict, reason code and evidence
parent_biosamples.parquet	16	Parent BioSamples of the run-level rows (provenance only; excluded from every count)
study_verdict_history.parquet	32,231	Every triage-stage verdict per study (screen, rubric, Sonnet replicates, Opus adjudication, growth waves, BioSample re-judge, human-review rounds, final) with model, confidence, evidence and source artifact
value_history.parquet	45,591	Determinations that are not current: superseded, rejected, dropped by the group-statement audit, duplicates, moved parent rows, auditor findings — each with status, reason, replaced_by
sample_determinations_superseded.parquet	223	Subset of the above kept in the v1.1 layout (run-level supersessions + v1.2 age corrections)
confidence_tiers.csv	29	The engine confidence tiers: route × determiner × scope with the fixed confidence values they emit
tier_field_precision.csv	49	Empirical precision of each route × confidence tier per field against the cMD gold join (hi-res and full sets)
sample_unit_classification_by_study.csv	67	Per-study sample-unit class (A run-keyed, C technical multi-run, X cross-study BioSample)
human_review_queue.csv	52	Studies the pipeline could not decide, with the reason
field_coverage_summary.csv, study_field_coverage_matrix.csv		Coverage per field and per study × field
extraction_gold_eval_hires.csv		Precision/recall vs curatedMetagenomicData
sample_determinations_all.parquet	620,214	Current and retired determinations: sample_determinations columns + release_added, release_retired (null = current), package_added, retired_reason, retired_change_stage — the per-field value timeline — R2026.1
releases.csv	6	Release registry: one row per release id (package version, date, tags, DOI, headline counts, notes file) — R2026.1
RELEASE_NOTES_R2026.1.md		Generated diff report vs the previous package (studies/samples, coverage, verdict flips, findings, Sandpiper, gold, schema) — R2026.1
contribute_worklist.csv	439	Community-contribution worklist: one row per OPEN included/uncertain study (≥ 1 of the six fields age/delivery/feeding/preterm/antibiotics/probiotic below 0.5 coverage on catalog_scope), ranked by priority_score, with blocker_code, unlock_text and a prefilled contribution Issue URL — R2026.2
contribute_worklist_fields.csv	2,634	Study × field detail of the worklist (coverage, best recoverability tier, field-level blocker, evidence) — R2026.2
registry_studies.parquet	54,410	Registry tier: one row per ENA study of the human sh...

Data package data-v1.17.0

Microbiome Repo — data package v1.17.0 (2026-10-09)

One catalog, two tiers, all ages. The registry tables (registry_*) cover every public human shotgun-metagenome study in ENA/SRA/DDBJ — 64,369 studies, all body sites and ages — with archive-only classification (host, UBERON-anchored body sites, life stages, assay, access) and the harvested BioSample tier (615,944 BioSamples with normalised site / age / sex / country). The catalog tables (gut_*) are the curated tier: every registry study of the human gut, all ages — 2,893 studies, 583,315 samples, 3,825,853 sample × field values — with age at collection, sex, BMI, country, health condition (controlled vocabulary), antibiotic exposure, subject and timepoint recovered from archive attributes (R1), supplementary tables (R2), paper full text (R3) and abstracts (R4); every value carries a verbatim evidence quote (≤ 12 words), a labelled source, a route and a confidence. Coverage over all gut samples: age 28%, sex 32%, country 92%, health condition 48%, antibiotic exposure 19%.

Start here. gut_studies.parquet (one row per study: classification, papers, per-field coverage, age-category and health-condition mix, curated depth) → gut_sample_metadata_wide.parquet (one row per sample: every field with __route / __confidence, age_category + basis, body_site_class) → gut_sample_determinations.parquet (the evidence behind every value). registry_studies.parquet is the registry; registry_runs_v1.17.0.parquet (every run) ships as a GitHub Release asset.

Infant extension. The 389 infant-gut studies were curated in greater depth in earlier releases (delivery mode, feeding mode, preterm status, gestational age, birth weight, maternal antibiotics, probiotics, HMO supplementation, NEC). Those rows are present in the gut_* tables (curated_source = infant_catalog, infant_scope = true reproduces the former infant catalog exactly: 72,358 samples) and the original deep tables (sample_metadata_wide, sample_determinations, study_metadata_wide, cohorts, universe_studies_all, sandpiper_*, …) remain in the package for reproducibility; the paragraphs and headline numbers of that extension are kept below under Infant extension (historical description).

Infant extension (historical description)

The registry_* tables cover every public human shotgun-metagenome study in ENA/SRA/DDBJ (all body sites, all ages; archive-only classification plus the harvested BioSample tier). All other tables belong to the first curated scope, the Infant Gut Shotgun-Metagenome Catalog — the description below is that scope's.

A curated, evidence-linked catalog of every public shotgun-metagenome study of the human infant gut (0–36 months) in the INSDC archives (ENA/SRA/DDBJ), with per-sample metadata recovered from archive attributes, supplementary tables and papers.

Headline numbers (computed from the tables in this package; see CHANGELOG v1.2 and v1.2.2). 389 included studies · 154,206 catalog samples (153,685 BioSample units + 521 run units from 6 one-BioSample-per-infant deposits; their 16 parent BioSamples are listed in parent_biosamples.parquet and counted nowhere else) · 174,022 sequencing runs · 618,898 sample × field determinations, each with a verbatim evidence quote (9,559 of them are out-of-scope ages that document adult_age_flag) · age scope: 67,598 samples with per-sample or per-subject infant evidence + 7,097 in studies triage judged all-infant = 74,695 age-scope infant samples, of which 72,358 are also gut/unknown body site = catalog_scope, the headline denominator (2,900 age-scope infant rows are body-site excluded/linked: skin, milk, oral, mother…); 14,410 samples sit in mixed-age deposits and 49,556 in studies without a per-study infant estimate (age unknown); 9,559 carry an out-of-range age (adult_flagged); 5,986 are mother/other samples · age at collection ≤ 1,100 d recovered for 53,669 of 74,695 age-scope infant samples (72%) for 52,649 of 72,358 catalog-scope samples (73%) and for 62,230 of 146,336 body-site infant-scope samples (43%; v1.2.0 said 54,105 of 146,631 before the v1.2.2 host/isolate exclusions; the v1.1 figure 53,802 was computed before the run-unit fix and the site's 54,150 included the 16 parent rows and rows later superseded) · precision vs curatedMetagenomicData (hi-res set, 3,498 gold rows; extraction_gold_eval_hires.csv): age 0.99, delivery mode 0.995, preterm status 1.00 (full set of 3,670 rows: age 0.976, delivery 0.995, preterm 1.00 — see tier_field_precision.csv, column gold_set).

Universe screened: 9,579 candidate studies (all METAGENOMIC WGS/WXS runs in ENA with a human signal, no taxon filter) → 389 included, 52 unresolvable after two full-text review rounds; recall on the 22 curatedMetagenomicData infant BioProjects: 17/22 by the automated cascade, 18/22 after the human-review pass (one study, decision_stage=session_model_review), the rest excluded with a coded reason (universe_studies_all.parquet).

Catalog membership: a study is in the catalog when it is in gut_studies.parquet (samples: gut_sample_metadata_wide.parquet). The catalog_status / triage_verdict / reason_code columns of universe_studies_all.parquet and study_metadata_wide.parquet are the historical INFANT-catalog screen (389 included) — do not filter the all-age catalog on them.

Files

File	Rows	What it is
sample_metadata_wide.parquet / .csv.gz	154,206	Start here. One row per sample: identifiers, body-site class, every metadata field with <field>__confidence and <field>__route, subject/timepoint, run accessions, cohort, links
sample_determinations.parquet	618,898	Long form: one row per sample × field with evidence_source, evidence_locator, evidence_quote (≤12 words verbatim), route, scope, confidence
study_metadata_wide.parquet / .csv	389	One row per included study (BioProject): title, counts, triage evidence, recoverability tiers, per-field coverage (cov_*), cohort, links
runs.parquet	174,022	Run → sample → study with library/instrument fields (join key into ENA/SRA)
sample_subjects.parquet	154,206	Subject and timepoint resolution per sample (subject_key, role infant/mother/other, t_index)
cohorts.csv	373	Cohort clusters (studies + papers sharing a cohort) with unique-infant estimates
study_paper_links.csv	973	Study ↔ paper links with PMID/PMCID/DOI and link method
sandpiper_flag_table.csv	20	Single source for every Sandpiper flag (field, level, site label, definition, used) — v1.2.2
sandpiper_study_panel_status.csv	389	Per-study panel scope label and n breakdown (age_scope of profiled rows) — v1.2.1
sandpiper_auditor_candidate_runs.csv	375	Runs whose organism label is a non-human host or a named microbe, with the v1.2.1 action taken — v1.2.1
review_genomic_only_samples.csv, review_named_microbe_metagenomic_samples.csv, review_amplicon_runs.csv	3158 / 70 / 39	Owner review lists (library_source GENOMIC on every run; METAGENOMIC runs with a named-microbe organism; AMPLICON-strategy runs) — v1.2.1
universe_studies_all.parquet	9,579	Every screened study with verdict, reason code and evidence
parent_biosamples.parquet	16	Parent BioSamples of the run-level rows (provenance only; excluded from every count)
study_verdict_history.parquet	32,231	Every triage-stage verdict per study (screen, rubric, Sonnet replicates, Opus adjudication, growth waves, BioSample re-judge, human-review rounds, final) with model, confidence, evidence and source artifact
value_history.parquet	45,591	Determinations that are not current: superseded, rejected, dropped by the group-statement audit, duplicates, moved parent rows, auditor findings — each with status, reason, replaced_by
sample_determinations_superseded.parquet	223	Subset of the above kept in the v1.1 layout (run-level supersessions + v1.2 age corrections)
confidence_tiers.csv	29	The engine confidence tiers: route × determiner × scope with the fixed confidence values they emit
tier_field_precision.csv	49	Empirical precision of each route × confidence tier per field against the cMD gold join (hi-res and full sets)
sample_unit_classification_by_study.csv	67	Per-study sample-unit class (A run-keyed, C technical multi-run, X cross-study BioSample)
human_review_queue.csv	52	Studies the pipeline could not decide, with the reason
field_coverage_summary.csv, study_field_coverage_matrix.csv		Coverage per field and per study × field
extraction_gold_eval_hires.csv		Precision/recall vs curatedMetagenomicData
sample_determinations_all.parquet	620,214	Current and retired determinations: sample_determinations columns + release_added, release_retired (null = current), package_added, retired_reason, retired_change_stage — the per-field value timeline — R2026.1
releases.csv	6	Release registry: one row per release id (package version, date, tags, DOI, headline counts, notes file) — R2026.1
RELEASE_NOTES_R2026.1.md		Generated diff report vs the previous package (studies/samples, coverage, verdict flips, findings, Sandpiper, gold, schema) — R2026.1
contribute_worklist.csv	439	Community-contribution worklist: one row per OPEN included/uncertain study (≥ 1 of the six fields age/delivery/feeding/preterm/antibiotics/probiotic below 0.5 coverage on catalog_scope), ranked by priority_score, with blocker_code, unlock_text and a prefilled contribution Issue URL — R2026.2
contribute_worklist_fields.csv	2,634	Study × field detail of the worklist (coverage, best recoverability tier, field-level blocker, evidence) — R2026.2
registry_studies.parquet	54,410	Registry tier: one row per ENA study of the human sh...

Data package data-v1.16.0

Microbiome Repo — data package v1.16.0 (2026-10-08)

One catalog, two tiers, all ages. The registry tables (registry_*) cover every public human shotgun-metagenome study in ENA/SRA/DDBJ — 64,369 studies, all body sites and ages — with archive-only classification (host, UBERON-anchored body sites, life stages, assay, access) and the harvested BioSample tier (615,944 BioSamples with normalised site / age / sex / country). The catalog tables (gut_*) are the curated tier: every registry study of the human gut, all ages — 2,893 studies, 583,315 samples, 3,351,031 sample × field values — with age at collection, sex, BMI, country, health condition (controlled vocabulary), antibiotic exposure, subject and timepoint recovered from archive attributes (R1), supplementary tables (R2), paper full text (R3) and abstracts (R4); every value carries a verbatim evidence quote (≤ 12 words), a labelled source, a route and a confidence. Coverage over all gut samples: age 26%, sex 31%, country 91%, health condition 40%, antibiotic exposure 17%.

Start here. gut_studies.parquet (one row per study: classification, papers, per-field coverage, age-category and health-condition mix, curated depth) → gut_sample_metadata_wide.parquet (one row per sample: every field with __route / __confidence, age_category + basis, body_site_class) → gut_sample_determinations.parquet (the evidence behind every value). registry_studies.parquet is the registry; registry_runs_v1.16.0.parquet (every run) ships as a GitHub Release asset.

Infant extension. The 389 infant-gut studies were curated in greater depth in earlier releases (delivery mode, feeding mode, preterm status, gestational age, birth weight, maternal antibiotics, probiotics, HMO supplementation, NEC). Those rows are present in the gut_* tables (curated_source = infant_catalog, infant_scope = true reproduces the former infant catalog exactly: 72,358 samples) and the original deep tables (sample_metadata_wide, sample_determinations, study_metadata_wide, cohorts, universe_studies_all, sandpiper_*, …) remain in the package for reproducibility; the paragraphs and headline numbers of that extension are kept below under Infant extension (historical description).

Infant extension (historical description)

The registry_* tables cover every public human shotgun-metagenome study in ENA/SRA/DDBJ (all body sites, all ages; archive-only classification plus the harvested BioSample tier). All other tables belong to the first curated scope, the Infant Gut Shotgun-Metagenome Catalog — the description below is that scope's.

A curated, evidence-linked catalog of every public shotgun-metagenome study of the human infant gut (0–36 months) in the INSDC archives (ENA/SRA/DDBJ), with per-sample metadata recovered from archive attributes, supplementary tables and papers.

Headline numbers (computed from the tables in this package; see CHANGELOG v1.2 and v1.2.2). 389 included studies · 154,206 catalog samples (153,685 BioSample units + 521 run units from 6 one-BioSample-per-infant deposits; their 16 parent BioSamples are listed in parent_biosamples.parquet and counted nowhere else) · 174,022 sequencing runs · 618,898 sample × field determinations, each with a verbatim evidence quote (9,559 of them are out-of-scope ages that document adult_age_flag) · age scope: 67,598 samples with per-sample or per-subject infant evidence + 7,097 in studies triage judged all-infant = 74,695 age-scope infant samples, of which 72,358 are also gut/unknown body site = catalog_scope, the headline denominator (2,900 age-scope infant rows are body-site excluded/linked: skin, milk, oral, mother…); 14,410 samples sit in mixed-age deposits and 49,556 in studies without a per-study infant estimate (age unknown); 9,559 carry an out-of-range age (adult_flagged); 5,986 are mother/other samples · age at collection ≤ 1,100 d recovered for 53,669 of 74,695 age-scope infant samples (72%) for 52,649 of 72,358 catalog-scope samples (73%) and for 62,230 of 146,336 body-site infant-scope samples (43%; v1.2.0 said 54,105 of 146,631 before the v1.2.2 host/isolate exclusions; the v1.1 figure 53,802 was computed before the run-unit fix and the site's 54,150 included the 16 parent rows and rows later superseded) · precision vs curatedMetagenomicData (hi-res set, 3,498 gold rows; extraction_gold_eval_hires.csv): age 0.99, delivery mode 0.995, preterm status 1.00 (full set of 3,670 rows: age 0.976, delivery 0.995, preterm 1.00 — see tier_field_precision.csv, column gold_set).

Universe screened: 9,579 candidate studies (all METAGENOMIC WGS/WXS runs in ENA with a human signal, no taxon filter) → 389 included, 52 unresolvable after two full-text review rounds; recall on the 22 curatedMetagenomicData infant BioProjects: 17/22 by the automated cascade, 18/22 after the human-review pass (one study, decision_stage=session_model_review), the rest excluded with a coded reason (universe_studies_all.parquet).

Catalog membership: a study is in the catalog when it is in gut_studies.parquet (samples: gut_sample_metadata_wide.parquet). The catalog_status / triage_verdict / reason_code columns of universe_studies_all.parquet and study_metadata_wide.parquet are the historical INFANT-catalog screen (389 included) — do not filter the all-age catalog on them.

Files

File	Rows	What it is
sample_metadata_wide.parquet / .csv.gz	154,206	Start here. One row per sample: identifiers, body-site class, every metadata field with <field>__confidence and <field>__route, subject/timepoint, run accessions, cohort, links
sample_determinations.parquet	618,898	Long form: one row per sample × field with evidence_source, evidence_locator, evidence_quote (≤12 words verbatim), route, scope, confidence
study_metadata_wide.parquet / .csv	389	One row per included study (BioProject): title, counts, triage evidence, recoverability tiers, per-field coverage (cov_*), cohort, links
runs.parquet	174,022	Run → sample → study with library/instrument fields (join key into ENA/SRA)
sample_subjects.parquet	154,206	Subject and timepoint resolution per sample (subject_key, role infant/mother/other, t_index)
cohorts.csv	373	Cohort clusters (studies + papers sharing a cohort) with unique-infant estimates
study_paper_links.csv	973	Study ↔ paper links with PMID/PMCID/DOI and link method
sandpiper_flag_table.csv	20	Single source for every Sandpiper flag (field, level, site label, definition, used) — v1.2.2
sandpiper_study_panel_status.csv	389	Per-study panel scope label and n breakdown (age_scope of profiled rows) — v1.2.1
sandpiper_auditor_candidate_runs.csv	375	Runs whose organism label is a non-human host or a named microbe, with the v1.2.1 action taken — v1.2.1
review_genomic_only_samples.csv, review_named_microbe_metagenomic_samples.csv, review_amplicon_runs.csv	3158 / 70 / 39	Owner review lists (library_source GENOMIC on every run; METAGENOMIC runs with a named-microbe organism; AMPLICON-strategy runs) — v1.2.1
universe_studies_all.parquet	9,579	Every screened study with verdict, reason code and evidence
parent_biosamples.parquet	16	Parent BioSamples of the run-level rows (provenance only; excluded from every count)
study_verdict_history.parquet	32,231	Every triage-stage verdict per study (screen, rubric, Sonnet replicates, Opus adjudication, growth waves, BioSample re-judge, human-review rounds, final) with model, confidence, evidence and source artifact
value_history.parquet	45,591	Determinations that are not current: superseded, rejected, dropped by the group-statement audit, duplicates, moved parent rows, auditor findings — each with status, reason, replaced_by
sample_determinations_superseded.parquet	223	Subset of the above kept in the v1.1 layout (run-level supersessions + v1.2 age corrections)
confidence_tiers.csv	29	The engine confidence tiers: route × determiner × scope with the fixed confidence values they emit
tier_field_precision.csv	49	Empirical precision of each route × confidence tier per field against the cMD gold join (hi-res and full sets)
sample_unit_classification_by_study.csv	67	Per-study sample-unit class (A run-keyed, C technical multi-run, X cross-study BioSample)
human_review_queue.csv	52	Studies the pipeline could not decide, with the reason
field_coverage_summary.csv, study_field_coverage_matrix.csv		Coverage per field and per study × field
extraction_gold_eval_hires.csv		Precision/recall vs curatedMetagenomicData
sample_determinations_all.parquet	620,214	Current and retired determinations: sample_determinations columns + release_added, release_retired (null = current), package_added, retired_reason, retired_change_stage — the per-field value timeline — R2026.1
releases.csv	6	Release registry: one row per release id (package version, date, tags, DOI, headline counts, notes file) — R2026.1
RELEASE_NOTES_R2026.1.md		Generated diff report vs the previous package (studies/samples, coverage, verdict flips, findings, Sandpiper, gold, schema) — R2026.1
contribute_worklist.csv	439	Community-contribution worklist: one row per OPEN included/uncertain study (≥ 1 of the six fields age/delivery/feeding/preterm/antibiotics/probiotic below 0.5 coverage on catalog_scope), ranked by priority_score, with blocker_code, unlock_text and a prefilled contribution Issue URL — R2026.2
contribute_worklist_fields.csv	2,634	Study × field detail of the worklist (coverage, best recoverability tier, field-level blocker, evidence) — R2026.2
registry_studies.parquet	54,410	Registry tier: one row per ENA study of the human sh...

Data package data-v1.15.0

Microbiome Repo — data package v1.15.0 (2026-10-08)

One catalog, two tiers, all ages. The registry tables (registry_*) cover every public human shotgun-metagenome study in ENA/SRA/DDBJ — 64,369 studies, all body sites and ages — with archive-only classification (host, UBERON-anchored body sites, life stages, assay, access) and the harvested BioSample tier (615,335 BioSamples with normalised site / age / sex / country). The catalog tables (gut_*) are the curated tier: every registry study of the human gut, all ages — 2,887 studies, 582,841 samples, 3,212,545 sample × field values — with age at collection, sex, BMI, country, health condition (controlled vocabulary), antibiotic exposure, subject and timepoint recovered from archive attributes (R1), supplementary tables (R2), paper full text (R3) and abstracts (R4); every value carries a verbatim evidence quote (≤ 12 words), a labelled source, a route and a confidence. Coverage over all gut samples: age 26%, sex 31%, country 91%, health condition 38%, antibiotic exposure 17%.

Start here. gut_studies.parquet (one row per study: classification, papers, per-field coverage, age-category and health-condition mix, curated depth) → gut_sample_metadata_wide.parquet (one row per sample: every field with __route / __confidence, age_category + basis, body_site_class) → gut_sample_determinations.parquet (the evidence behind every value). registry_studies.parquet is the registry; registry_runs_v1.15.0.parquet (every run) ships as a GitHub Release asset.

Infant extension. The 389 infant-gut studies were curated in greater depth in earlier releases (delivery mode, feeding mode, preterm status, gestational age, birth weight, maternal antibiotics, probiotics, HMO supplementation, NEC). Those rows are present in the gut_* tables (curated_source = infant_catalog, infant_scope = true reproduces the former infant catalog exactly: 72,358 samples) and the original deep tables (sample_metadata_wide, sample_determinations, study_metadata_wide, cohorts, universe_studies_all, sandpiper_*, …) remain in the package for reproducibility; the paragraphs and headline numbers of that extension are kept below under Infant extension (historical description).

Infant extension (historical description)

The registry_* tables cover every public human shotgun-metagenome study in ENA/SRA/DDBJ (all body sites, all ages; archive-only classification plus the harvested BioSample tier). All other tables belong to the first curated scope, the Infant Gut Shotgun-Metagenome Catalog — the description below is that scope's.

A curated, evidence-linked catalog of every public shotgun-metagenome study of the human infant gut (0–36 months) in the INSDC archives (ENA/SRA/DDBJ), with per-sample metadata recovered from archive attributes, supplementary tables and papers.

Headline numbers (computed from the tables in this package; see CHANGELOG v1.2 and v1.2.2). 389 included studies · 154,206 catalog samples (153,685 BioSample units + 521 run units from 6 one-BioSample-per-infant deposits; their 16 parent BioSamples are listed in parent_biosamples.parquet and counted nowhere else) · 174,022 sequencing runs · 618,898 sample × field determinations, each with a verbatim evidence quote (9,559 of them are out-of-scope ages that document adult_age_flag) · age scope: 67,598 samples with per-sample or per-subject infant evidence + 7,097 in studies triage judged all-infant = 74,695 age-scope infant samples, of which 72,358 are also gut/unknown body site = catalog_scope, the headline denominator (2,900 age-scope infant rows are body-site excluded/linked: skin, milk, oral, mother…); 14,410 samples sit in mixed-age deposits and 49,556 in studies without a per-study infant estimate (age unknown); 9,559 carry an out-of-range age (adult_flagged); 5,986 are mother/other samples · age at collection ≤ 1,100 d recovered for 53,669 of 74,695 age-scope infant samples (72%) for 52,649 of 72,358 catalog-scope samples (73%) and for 62,230 of 146,336 body-site infant-scope samples (43%; v1.2.0 said 54,105 of 146,631 before the v1.2.2 host/isolate exclusions; the v1.1 figure 53,802 was computed before the run-unit fix and the site's 54,150 included the 16 parent rows and rows later superseded) · precision vs curatedMetagenomicData (hi-res set, 3,498 gold rows; extraction_gold_eval_hires.csv): age 0.99, delivery mode 0.995, preterm status 1.00 (full set of 3,670 rows: age 0.976, delivery 0.995, preterm 1.00 — see tier_field_precision.csv, column gold_set).

Universe screened: 9,579 candidate studies (all METAGENOMIC WGS/WXS runs in ENA with a human signal, no taxon filter) → 389 included, 52 unresolvable after two full-text review rounds; recall on the 22 curatedMetagenomicData infant BioProjects: 17/22 by the automated cascade, 18/22 after the human-review pass (one study, decision_stage=session_model_review), the rest excluded with a coded reason (universe_studies_all.parquet).

Files

File	Rows	What it is
sample_metadata_wide.parquet / .csv.gz	154,206	Start here. One row per sample: identifiers, body-site class, every metadata field with <field>__confidence and <field>__route, subject/timepoint, run accessions, cohort, links
sample_determinations.parquet	618,898	Long form: one row per sample × field with evidence_source, evidence_locator, evidence_quote (≤12 words verbatim), route, scope, confidence
study_metadata_wide.parquet / .csv	389	One row per included study (BioProject): title, counts, triage evidence, recoverability tiers, per-field coverage (cov_*), cohort, links
runs.parquet	174,022	Run → sample → study with library/instrument fields (join key into ENA/SRA)
sample_subjects.parquet	154,206	Subject and timepoint resolution per sample (subject_key, role infant/mother/other, t_index)
cohorts.csv	373	Cohort clusters (studies + papers sharing a cohort) with unique-infant estimates
study_paper_links.csv	973	Study ↔ paper links with PMID/PMCID/DOI and link method
sandpiper_flag_table.csv	20	Single source for every Sandpiper flag (field, level, site label, definition, used) — v1.2.2
sandpiper_study_panel_status.csv	389	Per-study panel scope label and n breakdown (age_scope of profiled rows) — v1.2.1
sandpiper_auditor_candidate_runs.csv	375	Runs whose organism label is a non-human host or a named microbe, with the v1.2.1 action taken — v1.2.1
review_genomic_only_samples.csv, review_named_microbe_metagenomic_samples.csv, review_amplicon_runs.csv	3158 / 70 / 39	Owner review lists (library_source GENOMIC on every run; METAGENOMIC runs with a named-microbe organism; AMPLICON-strategy runs) — v1.2.1
universe_studies_all.parquet	9,579	Every screened study with verdict, reason code and evidence
parent_biosamples.parquet	16	Parent BioSamples of the run-level rows (provenance only; excluded from every count)
study_verdict_history.parquet	32,231	Every triage-stage verdict per study (screen, rubric, Sonnet replicates, Opus adjudication, growth waves, BioSample re-judge, human-review rounds, final) with model, confidence, evidence and source artifact
value_history.parquet	45,591	Determinations that are not current: superseded, rejected, dropped by the group-statement audit, duplicates, moved parent rows, auditor findings — each with status, reason, replaced_by
sample_determinations_superseded.parquet	223	Subset of the above kept in the v1.1 layout (run-level supersessions + v1.2 age corrections)
confidence_tiers.csv	29	The engine confidence tiers: route × determiner × scope with the fixed confidence values they emit
tier_field_precision.csv	49	Empirical precision of each route × confidence tier per field against the cMD gold join (hi-res and full sets)
sample_unit_classification_by_study.csv	67	Per-study sample-unit class (A run-keyed, C technical multi-run, X cross-study BioSample)
human_review_queue.csv	52	Studies the pipeline could not decide, with the reason
field_coverage_summary.csv, study_field_coverage_matrix.csv		Coverage per field and per study × field
extraction_gold_eval_hires.csv		Precision/recall vs curatedMetagenomicData
sample_determinations_all.parquet	620,214	Current and retired determinations: sample_determinations columns + release_added, release_retired (null = current), package_added, retired_reason, retired_change_stage — the per-field value timeline — R2026.1
releases.csv	6	Release registry: one row per release id (package version, date, tags, DOI, headline counts, notes file) — R2026.1
RELEASE_NOTES_R2026.1.md		Generated diff report vs the previous package (studies/samples, coverage, verdict flips, findings, Sandpiper, gold, schema) — R2026.1
contribute_worklist.csv	439	Community-contribution worklist: one row per OPEN included/uncertain study (≥ 1 of the six fields age/delivery/feeding/preterm/antibiotics/probiotic below 0.5 coverage on catalog_scope), ranked by priority_score, with blocker_code, unlock_text and a prefilled contribution Issue URL — R2026.2
contribute_worklist_fields.csv	2,634	Study × field detail of the worklist (coverage, best recoverability tier, field-level blocker, evidence) — R2026.2
registry_studies.parquet	54,410	Registry tier: one row per ENA study of the human shotgun-metagenome universe (all body sites, all ages) — host_human, assay, body_sites/life_stages (config/vocab codes with UBERON anchors), population flags, access, evidence rows, classification_stage, infant-catalog verdict, scope_memberships (config/scope.yaml) — R2026.4
registry_universe_audit.csv	35	Registry enumeration audit: per ENA slice the archive c...