Repository navigation
v4.0.0 — the geometry half on one pass over one matrix of repertoires
Signatures rewritten from scratch. The geometry half; the statistics half is vdjtools, which
this release requires at >=4.0.1 -- the rsig fit and artifact I/O live in
vdjtools.signature.corpus, which does not exist before 4.0.0, and 4.0.0 never reached PyPI.
No legacy path, no backward compatibility, no artifact carried over.
Performance
synthesize now runs on the shared vdjtools.signature.corpus.build_matrices, so the rsig corpus
is built across samples in worker processes: each worker draws its own repertoires from
memory-mapped pools and returns one row of numbers. The artifact is bit-identical at every n_jobs,
because a sample is a pure function of its index. mir corpus -j/--jobs exposes it -- worker
processes, 0 = the whole allocation; the help text says which layer it reaches.
Measured on a 16-core M-series laptop, one seven-locus sample of 10,000 clonotypes per locus:
raw_and_channels is 0.136 s, so the rsig row was never the expensive part of a build. What was
expensive is receptor generation, which is shared with vdjtools and is now parallel there.
The defect this fixes
build_rsig.py fitted R_V, R_J and R_C on (10_000, 768) — one row per clonotype, from
the bundled prototype panel — while every one of the 399 PC columns it produced was a repertoire
statistic, obtained by projecting a clone-weighted mean through those axes. PCA over 10,000
receptors evaluated at a mean is not PCA of repertoires: the two maximise variance in different units
and give different axes. A sample's coordinate averages ~400 effective clones, so variance between
samples along those axes is of order 1/sqrt(400) of the variance being maximised.
The artifact's centre and scale were fitted on zero rows and arrived from a separate corpus of
real samples. Two independent fits, stitched — which is how blood came to pair a centre of exactly
0.0 with a scale plainly fitted from data on 64 rsig:contrast columns, putting a corpus-typical
sample 81 robust deviations out with 885 of 885 samples of an independent cohort outside the
bound; and how tissue came to carry blood's coverage constants for all seven loci to 17
significant digits while its own location and scale had genuinely been refitted.
Rotation, bounds, centre and scale now all come out of one pass over one matrix of repertoires,
per locus, using the same vdjtools.signature.corpus module that fits the statistics half — so the
two cannot drift into different notions of "standardised".
Added
mir.signature.features—Phiand its functionals, a pure function of one sample plus the
bundled prototype panel.mir.signature.signature—rsig,rsig_cohort,raw_and_channels,synthesize. It also
declares the rsig half of the column contract, registering into vdjtools' layout at import.mir corpus— build and fit a corpus. Draws the same synthetic repertoires from the same seeds as
vdjtools corpus, so the two halves of a corpus name describe the same repertoires.rsig:qc:-:winsor_frac— what fraction of a row the corpus's bounds clamped. Never silent.
Removed
signature/assemble.py,blocks.py,reference.py,scale.py,
resources/signature/build_rsig.py, and all 16 bundled artifacts (8.4 MB: six scale
references,rsig_v2.npz,kmer_spaces_v1.npz).- The
contrastgroup — 231 columns — and nothing was lost. It wasPsi = mass*(Phi - naive)
withnaivea separately drawn 20,000-sequence reference, because the rotation was fit-free and
needed an explicit subtraction point. The corpus centre now is that point: rotating through the
naivecorpus subtracts the medianPhiof unselected repertoires. One frozen vector and one
whole failure mode — anaivedrawn against a different release of the recombination models, which
moved every contrast column by 0.1–1.6% per locus — replaced by choosing a corpus.massremains a
feature, so the rotation still sees it. signature/signature_cohort. Each half has its own artifact now, so the wrapper's only real
job — one scale reference over both — no longer exists. Two calls and a polars join, and the halves
are disjoint by construction so the join cannot collide.- Tiers, and the
mir presetscommand.--componentsreplaces both. - CLI flags
--tier,--preset,--channels,--standardize,--scale,--clip,--squash,
--on-unscaled,--threads. Each is pinned by a test asserting a non-zero exit. examples/feature_vectors.py, whose entire subject was presets.
Fixed
- A one-part composition crashed rather than holing. When every clone-size compartment falls
belowmin_clonotypes, only the closing residual is left — which is no composition at all, so
there are no ratios to take.clrraised on one part; the coordinates are nownan, and the
geometry itself is still measurable beside them.
Measured while building this
- 5 components reach 0.91–0.93 of the variance on a small TRG/TRD corpus, against 0.34–0.48 for
the same count on the statistics half. The 256Phicoordinates are correlated distances to one
panel, so this half is far more compressible — worth knowing before picking--components. - Rao agrees with
mir.repertoire.rao_dispersionto 1e-9 relative, cross-checked rather than
trusted: two spellings of one telescoping identity. - The corpus build is byte-identical across thread counts, verified
cmpon the npz.
Added: the shipped corpora, and a gate on the germline they were drawn from
rsig_naive and rsig_memory ship in src/mir/resources/signature/ — 10,000 synthetic repertoires
each at 10,000 receptors, all seven human loci, k = 128 per locus, 1.62 MB. Variance reached at 128
components runs from 0.9887 (IGH, naive) to 0.9992 (IGK), against 0.377–0.854 for the
statistics half, because the geometry half's raw features are a few hundred emb