Step 4 Reports and Outputs

Step 4: Reports & Outputs

What is this?

Everything a run produces: the trees, the surveillance report for the lab, and the run report about the run itself.

Why does it matter?

The surveillance report is what the lab reads; the run report and provenance let anyone trace a number back to the exact data, code and tools that made it.

What do I do?

Nothing to trigger them: both reports are built at the end of every run. Open them, read them, and keep them confidential.

flowchart LR
    T["auspice/*.json<br/>27 builds"] --> SR["surveillance_report rule<br/>scripts/report.py"]
    FL["results/*/*/filter_log.tsv"] --> SR
    M["source/ metadata +<br/>flusort typing"] --> SR
    SR --> R["reports/<br/>HTML, PowerPoint,<br/>figures, tables"]
    LOGS[".run/logs + benchmarks<br/>+ provenance"] --> RR["end-of-run handler<br/>scripts/run_report.py"]
    RR --> RUN[".run/reports/{RUN_ID}/"]

Loading

Two reports, two questions: "what did we sequence?" (surveillance report) and "how did this run go on this machine?" (run report).

Where outputs land

Output	Location	Share?
Trees (one per build + tip frequencies)	auspice/{subtype}/{segment}.json, {segment}_tip-frequencies.json	Only by upload after review (Step 5)
Surveillance report	reports/	Confidential, named internal recipients
Filter logs, Nextclade tables, alignments, trees	results/{subtype}/{segment}/	Never (contain GISAID-derived data)
Run logs, provenance, run reports	.run/	Internal (contain user and host names)
Archives	snapshots/	Never outside the lab

Surveillance report

Built by the surveillance_report rule in workflow/snakemake_rules/reports.smk with one script, scripts/report.py (it replaced build-reports.py and render-reports.qmd; Quarto and R are no longer needed). It complements the Nextstrain build: "submitted" always means specimens fed into the pipeline (rows of its input metadata), not every specimen collected or sequenced.

Output (reports/)	What it is
surveillance_report.html	Interactive, one self-contained file; opens offline in any browser. Charts have tooltips and menus; tables search, sort, filter and export to CSV
surveillance_report.pptx	Static slide deck of the same figures and tables
figures/*.png	Every figure at 2× resolution
tables/*.tsv, report.xlsx	Every table behind the report
report.tsv	One row per fludb sequence: type, subtype (the build subtype), subtype_flusort (flusort's call, e.g. xxN1), subtype_assumed, clade, segment completeness

How to read it (same order in HTML and deck):

1. At a glance: most recent run vs. season to date, submitted vs. in the final build.
2. Surveillance summary: the JHH Influenza Surveillance Summary and Genetic Characterization tables (Most recent run and Season to date). The HTML page has a season menu; the deck shows the current season plus one appendix slide per earlier season.
3. Type/subtype and clade/subclade frequency per sequencing run, with dotted lines at each Oct 1 and May 31 (a visual aid only, not true season boundaries).
4. Quality control: run-level funnel (in, removed at each step and why, in the final build; by subtype or type, all runs or one run), outcome of each run's specimens, QC summary per run (incl. HA only / HA + NA / whole genome), run-detail heatmap, retention per build, Nextclade QC metrics, searchable tables of every removed sequence and every specimen, and whole-database figures. Filter details: Step 3.
5. Interactive tables and Methods.

Seasons: October 1 to about May 31. From October onward the current season starts October 1 of this year, otherwise October 1 of last year (so June to September still show the season that just ended).

Subtype assumed: a specimen with only HA or only NA called by flusort (H1xx, xxN1, H3xx, xxN2) is counted as H1N1 or H3N2 and built as such (see Step 3). The report shows this as subtype assumed in the specimen journey, the per-run QC table (Subtype assumed column), the genome completeness table and the table captions, and the Methods end with a disclaimer explaining the assumption. Both the builds and the report read the rule from config/subtype_assignment.yaml, so a change there (e.g. if a reassortant circulates) changes both together.

Prepared by shows who ran the pipeline: the full name of the computer account that ran it (macOS, Linux or WSL), otherwise its login name. It is separate from the fixed "Developed by" attribution, which the build checks for and must not be removed.

Settings in config/report.yaml: title, subtitle, confidentiality notice, notes/callouts, plot date window, runs to leave out of plots, number of clades shown before "Other", the control-name pattern, QC thresholds (keep in sync with Step 3) and rows per slide.

python scripts/report.py                       # rebuild by hand (production inputs, source/)
python scripts/report.py --source-dir tutorial # tutorial inputs
python scripts/report.py --as-of 2025-03-01    # as it looked on that date
python scripts/report.py --help                # all options
marimo run scripts/report.py                   # live dashboard (date pickers, run explorer)
marimo edit scripts/report.py                  # notebook view, to add or change plots
snakemake --cores 8 --config surveillance_report=false   # skip it during a run

Caution

Build reports from the pekosz-nextstrain conda environment, not the Nextstrain Docker runtime. The Docker image may not include the extra packages the reports need (Altair, vl-convert, marimo, itables, python-pptx, openpyxl, matplotlib), so report generation may fail there. See Why conda and not Docker?.

Adding a plot or table: write an item_* function in scripts/report.py that returns a Chart or Table with title and caption, and list it in report_layout(). It then appears in the HTML report, the deck, the exports and the marimo app.

Note

snapshot_clean archives and removes reports/: build or rebuild reports before cleaning. The report is confidential and its tables include the vaccine strains' GISAID IDs, so it is never posted publicly.

Run report

Everything a run records about itself is under one hidden folder, .run/:

.run/
├── logs/{rule}/{subtype}_{segment}.log          # stdout + stderr of every job (genome builds: {subtype}.log)
├── benchmarks/{rule}/{subtype}_{segment}.tsv    # runtime, peak memory and CPU per job
├── run_logs/
│   ├── {RUN_ID}_provenance.txt                  # start/finish (UTC), host, user, git commit + dirty files, tool versions,
│   │                                            # input files (records, size, modified, SHA-256), Nextclade dataset tags
│   └── {RUN_ID}_snakemake_{SUCCESS,FAILED}.log  # copy of Snakemake's main log
└── reports/
    ├── index.md                                 # every run: status, start, wall clock, jobs, commit
    └── {RUN_ID}/
        ├── report.md                            # summary document for the run
        ├── figures/                             # timeline, runtime by rule/build, memory, run history (PNG)
        └── tables/                              # jobs.tsv, rules.tsv, run_history.tsv

RUN_ID is the run's start time (YYYYMMDDTHHMMSS), so every file from one run shares a prefix. The run report is built at the end of every run, successful or failed, by scripts/run_report.py (called from the end-of-run handlers in reports.smk). It opens with the run's provenance, then summarises job timing, runtime and peak memory by rule and by build, failed jobs (with the tail of their log) and job logs that mention warnings or errors. Open report.md in a Markdown preview; figures are plain PNGs. It is separate from Snakemake's own --report output.

Job logs and benchmarks are overwritten when a job reruns, so a report only attributes them to a run if no later run re-executed that job. snapshot_clean archives and removes .run/logs/ and .run/benchmarks/, but keeps .run/run_logs/ and .run/reports/, so run history builds up.

python scripts/run_report.py                          # latest run
python scripts/run_report.py --run-id 20261006T191100 # a specific run
python scripts/run_report.py --all                    # every run in .run/run_logs/

What is recorded automatically

What	Where	When
Run ID	Every .run/ file name	Start time, YYYYMMDDTHHMMSS
Run provenance: start/finish (UTC), status, host, user, folder, git commit, uncommitted files, tool versions, conda env	.run/run_logs/{RUN_ID}_provenance.txt	Each run
Input files used (JHH_sequences.fasta, JHH_metadata.txt, vaccine.fasta): record count, size, last modified (UTC), SHA-256	.run/run_logs/{RUN_ID}_provenance.txt (== input files ==)	Run start
Snakemake main log	.run/run_logs/	Each run
Per-job logs and benchmarks	.run/logs/, .run/benchmarks/	Overwritten on rerun
Run report and run history	.run/reports/	Each run
Data scan and hook status	Console at run start	Each run
Which sequences each filter removed, and why	results/*/*/filter_log.tsv	Each build
Nextclade dataset version (tag) of every dataset used	.run/run_logs/{RUN_ID}_provenance.txt (== nextclade datasets ==); also .run/logs/fetch_hana_datasets/	Run end; HA/NA download
Build date	auspice/*.json (meta.updated)	Each export
Report date, season, "Prepared by"	reports/	Each report
Source prep date, who ran it, input checksums	source_prep/output/prep_report.txt	Each prepare_source.py run
Full archive of a run	snapshots/{YYYYMMDDTHHMMSS}.tar.gz	Each snapshot_clean
Code changes, author, date	git history	Each commit

Not recorded automatically (record by hand, Best Practices): when and how vaccine.fasta was downloaded from GISAID (the checksum in the provenance file identifies which file a run used, not where it came from), and who uploaded which build to Nextstrain and when.

Troubleshooting / FAQ

• The report step failed. Read .run/logs/surveillance_report/surveillance_report.log. If it says the attribution is missing, the report script was edited incorrectly; tell the maintainer.
• ModuleNotFoundError (altair, marimo, pptx, openpyxl...). You are probably inside the Nextstrain Docker runtime (nextstrain shell) or the conda environment is not active. Leave the Docker shell, run conda activate pekosz-nextstrain, then bash scripts/build_env.sh --check.
• "WARNING: run report failed". The run itself is fine; rerun python scripts/run_report.py --run-id <RUN_ID>.
• Numbers differ from last week's report. Check the as-of date and season, then exclude_runs_from_plots in config/report.yaml.
• I deleted reports/ by cleaning. It is inside the latest snapshots/*.tar.gz.

Next step → Step 5: Reviewing & Publishing Builds