Skip to content

Add porting functionality for drvi-py to scvi-tools migration#130

Merged
moinfar merged 3 commits into
mainfrom
forward_compatibility_for_drvi_on_scvi
Jul 8, 2026
Merged

Add porting functionality for drvi-py to scvi-tools migration#130
moinfar merged 3 commits into
mainfrom
forward_compatibility_for_drvi_on_scvi

Conversation

@moinfar

@moinfar moinfar commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Motivation

DRVI's PyTorch model has moved out of drvi-py into scvi-tools (scvi.external.DRVI). Users who trained models with drvi-py need a way to keep using them once drvi-py drops its module code (version >= 0.3.0). This PR adds a checkpoint-migration utility that converts a saved drvi-py model (model.pt) into one loadable by scvi.external.DRVI.load(...).

How to port

Minimal code:

import scvi
from drvi

new_dir = drvi.utils.port_to_scvi_tools("path/to/old_model", "path/to/new_model")
model = scvi.external.DRVI.load(new_dir, adata)
  • Pure checkpoint surgery. Reads the old model.pt dict, rewrites model_state_dict, init_params_ and registry_, and writes a new model.pt. It never instantiates an nn.Module and imports neither drvi-py's nor scvi-tools' model classes — so it keeps working even after drvi-py removes its module (only torch + packaging are needed).
  • Non-destructive. The source is never modified; an existing dest is never overwritten unless overwrite=True.

Additional information (🤖 Generated with Claude Code)

Key transformations

  • State-dict remap: StackedLinearLayer / split_transformation_weight layout (n_split, in, out) → scvi's (n_split, out, in) (transpose of every per-split weight); encoder mean/var un-nested from FCLayers to plain Linear; px_shared_decoder → px_decoder, params_nets.mean → px_scale_decoder; synthesized zeroed l_encoder.* (scvi's VAE always has a library encoder; strict load requires the keys, but they're unused since DRVI uses the observed library size).
  • Dispersion offset (r = 1.0 + param in drvi vs px_r used directly in scvi): folded correctly for gene, gene-batch, and gene-cell, matching drvi's runtime add-order for bit-exactness where possible.
  • Version migrations as a small chain (one function per transition, e.g. drvi_py_0.2.1 → 0.2.2 gene-likelihood renames / prior_init_obs removal, and the drvi_py → scvi_tools split-value renames). Adding a future breaking change is one function + one table entry.
  • Registry fixes so scvi's setup_anndata accepts it on load (drops is_count_data, adds size_factor_key, sets model_name="DRVI").
  • Capability gate: raises DRVIPortError for features scvi-tools DRVI cannot reproduce (learned covariate embeddings, non-normal prior, unsupported split methods/aggregations, non-uniform layer widths, decoder dropout, dropped gene-likelihoods); warns for training-only knobs that don't affect a trained model's forward pass. Every error message links users to open an issue if they need an unsupported case.

Verification

Validated end-to-end by loading each model in both frameworks and comparing the deterministic forward pass (latent, px.mean, px.log_prob, reconstruction) across 7 models — three real trained checkpoints (immune, HLCA, and a 261 MB / 40k-gene Parse-Bio model with mean_activation) plus trained models covering decoder_reuse_weights="nowhere", gene-batch, and gene-cell:

  • Bit-exact (Δ = 0.0) for all gene and gene-batch models.
  • gene-cell matches to ~1e-6 — an inherent last-ULP difference (drvi adds its +1 outside a nonlinear logsumexp, which checkpoint surgery can only fold inside); numerically identical distributions.

Unit tests in tests/utils/test_porting.py (6 tests) cover the key remaps, transposes, all three dispersion modes, no-overwrite behavior, and capability errors.

@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.65546% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.62%. Comparing base (dc7e8b3) to head (b0da8f2).

Files with missing lines Patch % Lines
src/drvi/utils/porting.py 88.60% 27 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #130      +/-   ##
==========================================
+ Coverage   82.00%   82.62%   +0.61%     
==========================================
  Files          46       47       +1     
  Lines        2823     3061     +238     
==========================================
+ Hits         2315     2529     +214     
- Misses        508      532      +24     
Files with missing lines Coverage Δ
src/drvi/utils/__init__.py 100.00% <100.00%> (ø)
src/drvi/utils/porting.py 88.60% <88.60%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@moinfar

moinfar commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator Author

This will resolve @cvduffy 's comment on scverse/scvi-tools#3866

@moinfar moinfar merged commit 8bfce79 into main Jul 8, 2026
9 checks passed
@moinfar moinfar deleted the forward_compatibility_for_drvi_on_scvi branch July 8, 2026 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant