Metagenome (re‐)generation

samovar regenerate

Two different “regenerate” steps share this name:

1. Abundance tables — rewrite annotator count tables (direct / bootstrap / vae / glm / optional R samovar).
2. Reads from those tables — ISS (or CAMISIM table→ISS) fetches genomes and simulates FASTQ (regenerated/).

CLI: samovar prepare --reannotation-level … writes .log/configs/config_annotation2iss.yaml. samovar exec runs the Snakemake rule process_annotations. Standalone tables: python -m samovar.table2iss / process_abundance_table.

Where files land

Path	What
$out/initial_annotations/*.annotation.csv	Combined tool calls (input to table regen)
$out/regenerated/.regenerated_abundance/	Per-annotator abundance CSVs after table regen
$out/genomes/	Run copy of FASTAs used for ISS (hardlinks when possible)
$SAMOVAR_DATABASE/processed/{accession}.fa.gz	Install library; main config genomes.data is updated as soon as a genome is fetched
$SAMOVAR_DATABASE/proteomes/{taxid}.faa.gz	Shared Kaiju proteome/GFF cache (not re-downloaded per DB)
$out/regenerated/{sample}_{annotator}_R1/R2.fastq	Simulated reads
$out/initial_trimmed/	QC of initial/ (identity when --qc is omitted)
$out/regenerated_trimmed/	QC of regenerated FASTQ before re-annotation

Optional read QC (samovar tools import --type QC) runs as checkpoints qc_initial and qc_generated. Default is identity (reads are already treated as trimmed). --qc applies to both stages; --qc-initial / --qc-generated may differ; --qc-postfix illumina:NAME selects a tool for hybrid {sample}_{postfix}_R* files. Annotators read the *_trimmed directories. See Custom tools import.

Kaiju/Kraken indexes are under databases (separate from the genome library). They should consume FASTAs already in genomes.data. After seed_genomes / regenerate_reads, exec also harvests $out/.genomes into the catalog so a later samovar build does not hit NCBI again for the same taxid.

Table regeneration (regeneration_mode)

Set in prepare YAML or workflow/annotation_regenerate.py:

Mode	Behaviour
direct (default; aliases preserve, exact)	Observed taxID counts, same sample names
bootstrap	Dirichlet-multinomial resample of columns
vae	FactorAnalysis on log abundances
glm	Correlation-aware synthetic communities (Python)
samovar	Optional R annotation_regenerate.R (not in the Python install)

--N_reads / N_reads is the depth of each regenerated sample (ISS), not the table mode. N is the number of synthetic samples for generative modes.

Genome lookup (--reannotation-level)

Default taxid / t: look up that NCBI taxID in the run genomes/ dir, then genomes.data, then download that taxid from NCBI.

Progressive ranks (catalog first, then NCBI under that id):

Flag	Alias	Search order after exact taxid
species	s	species (strain → species, or all strains of a species)
genus	g	species, then genus (up to genus of a species, or all indexed species of a genus)
family	f	species → genus → family
any	a	species → genus → family → order → class → phylum

A hit is ≥1 FASTA in the install catalog whose taxon at that rank matches (using nodes.dmp / merged.dmp). If none, Entrez txidN[Organism:exp] fetches one complete RefSeq assembly, processes it, and writes genomes.data immediately.

samovar prepare --output_dir RUN --reannotation-level genus \
  --kraken2-test "kraken2 phage_test"

Generate vs regenerate

samovar generate builds initial communities from a genome folder or --accessions (--reindex 0/1/2, see samovar_database.md).

samovar exec regenerate_reads builds a second community from annotation tables so ML/OPAL can compare tools against a known mix. It does not re-run CAMISIM unless you use --simulator camisim --camisim-mode table at generate time.