eidolon v3.4.0

eidolon v3.4.0 — a quality model measured from real, named data

eidolon's shipped sequencing-error model no longer comes from a sample nobody recorded. It is fitted from GIAB HG002 NIST_Illumina_2x250bps — public, downloadable, and documented down to the chunk, the sampling stride, the read count and the job that produced it.

📖 Documentation is now a site: https://ncsa.github.io/eidolon/

The default model is fitted from data you can download

Read length	250 bp
Quality scores	31 observed levels, Q2–Q40, continuous
Reads fitted	3,391,610 per mate, sampled across the file
error_rate	0.003774, against the inherited 0.006638

It carries both mates separately, because they are measurably different — 2.07x on error rate, 2.36x on degraded fraction.

Reads now degrade the way real reads do

About one read in ten in real Illumina data loses signal toward the 3' end and stays collapsed. eidolon's quality model had no per-read state to represent that and produced essentially none of it.

	real	v3.4.0	previous single-population model
R1, last 50 bases below Q25	11.12%	11.95%	0.54%
R1, below Q20	4.92%	5.91%	0.00%
R2, below Q25	26.22%	26.36%	11.74%
R2, below Q20	12.48%	14.00%	0.32%

Fixed: R2's quality scores were emitted backwards

This affected every paired FASTQ eidolon has ever produced. R2's per-cycle quality ran in reverse — reads improved toward the 3' end where real reads get worse — and because errors are injected from each base's own quality, R2's sequencing errors were mirrored with it.

Also

• Sequencing-error indels were ~40x too frequent and uniform across the genome; corrected and now scaled by local homopolymer run length.
• Model files carry a provenance stamp, so a model from a newer eidolon is refused rather than silently misread.
• gen-seq-error-model takes its read length from the reads rather than the first record, and validates FASTQ structurally as it reads.

Upgrading

• gen-reads' read_len now defaults to 250 to match the model. gen-cancer-reads is unchanged at 151.
• If you relied on that default and set fragment_mean, check it still suits 250 bp reads. Paired-end fragments have always had to exceed the read length; that is unchanged, and only the default moved. Configs from the shipped template set read_len explicitly and are unaffected.
• Output for a given seed changes, R1 included. Fragment placement is untouched, but a sequencing error consumes extra random draws, so changing which R2 bases error shifts the stream. Per-seed content has never been part of the public API.
• Peak RSS rises ~22 MB on the default path: four quality tensors where there was one.

Known and open

somatic_snv_recall reads 0.8397 against a 0.921 baseline and has since #660, now on four builds. This release eliminated the quality model as the cause — it replaced the entire model and the metric did not move. Tracked in #680.

The default is HiSeq 2500-era chemistry with continuous quality scores, not binned — the best provenanced option available from GIAB's HG002 path, not the most modern one. See #730.

---

Full detail in CHANGELOG.md. Model provenance in model_data/README.md.