Adapt all internal models to new setup #1301

justjhong · 2022-01-12T20:45:00Z

To adapt some models, this PR also adds an LabelsWithUnlabeledObsFIeld for PeakVI.

adamgayoso · 2022-01-14T06:09:03Z

scvi/data/anndata/fields/_scanvi.py

+    An AnnDataField for labels which include explicitly unlabeled cells.
+
+    Remaps unlabeled category to the final index if present in labels.


I would just better explain that unlabelled cells are labelled with a special category name that is user defined.

Labeled and labelled are both correct spellings. Labeled is the preferred spelling in American English. Labelled is the preferred spelling in British English.

Already integrating yourself into British culture I see

* adapt cellassign * adapt gimvi * adapt solo model * adapt stereoscope

* adapt LDA * adapt linearscvi * remove _get_var_names_from_setup_anndata * adapt peakvi * adapt autozi * adapt scanvi * fix scanvi test * fix totalvi test * fix dataloader tests * fix multiple cov tests * adapt condscvi * adapt destvi * adapt multivi * fix setup compat test * remove get_from_registry util * fix scanvi and peakvi scarches tests * fix backwards compat tests and default missing summary stat in models * address comment * Adapt all external models to new setup (#1302) * adapt cellassign * adapt gimvi * adapt solo model * adapt stereoscope

* adapt totalVI to new setup * fix additional totalvi test * remove run_setup_anndata kwarg, reassign uuid on view copy, fix some tests * remove run_setup_anndata comments * fix transfer anndata test * fix test data format test * fix setup anndata test * fix most pyro tests * fix most scarches tests * remove state dict from field * Adapt all internal models to new setup (#1301) * adapt LDA * adapt linearscvi * remove _get_var_names_from_setup_anndata * adapt peakvi * adapt autozi * adapt scanvi * fix scanvi test * fix totalvi test * fix dataloader tests * fix multiple cov tests * adapt condscvi * adapt destvi * adapt multivi * fix setup compat test * remove get_from_registry util * fix scanvi and peakvi scarches tests * fix backwards compat tests and default missing summary stat in models * address comment * Adapt all external models to new setup (#1302) * adapt cellassign * adapt gimvi * adapt solo model * adapt stereoscope

* placeholder release notes for base * AnnData Manager refactor (#1238) * move anndata file into anndata subfolder * mock out anndata manager and fields for categorical obs fields only * add layerfield * fix imports * fix imports in models * add temporary test and fix bugs in manager code * manager store on model class, refactor what needs to be defined for a model * clean up some existing code * restructure fields file, clean up more view anndata setup code, add more skeleton * add transfer_field for obs and layer fields * naive implementation of transfer_setup in anndata manager * implement backwards compat with setup dict on load * add docstrings to all new methods * address comments: change to registry_key, change fields to set in manager * add tests for transfer and manager from setup * remove anndata_fields method * address pr comment * Implement NonCategoricalJointObsField (#1247) * implement continuous obsm field * make class specific to joint obs fields * make fields public * address comments * faster concat by subsetting obs * Implement CategoricalJointObsField (#1248) * add categorical joint obs field * fix _make_obsm_categorical * add line in test * address pr comment * Pass manager to data loader (#1280) * pass manager to data loader * address comment * Move and reorganize setup dict out of AnnData into Manager (#1285) * new base field API for setup dict * adapt manager to new registry * backwards compatibility implementation * train works * adapt get_from_registry function * simplify needs transfer logic, working test_scvi * change _REGISTRY_KEYS back to _CONSTANTS * address comments, fix compat test * codacy * Add anndata registration classes to developer API (#1292) * add anndata registration classes to developer API * move manager to anndata submodule to fix docs * Save/Load with saved setup inputs (#1294) * skeleton for new save load, missing field logic * save load with saved setup inputs * add docstring for setup_inputs * codacy * fix documentation * additional pr comments * fix manager imports * Adapt TotalVI to new setup (#1298) * obsm field * adapt totalVI to new setup * fix additional totalvi test * make field registration ordered * fix totalvi protein priors logic * address pr comments * address pr comments * Fix tests for SCVI, TOTALVI, generic AnnData utils (#1299) * adapt totalVI to new setup * fix additional totalvi test * remove run_setup_anndata kwarg, reassign uuid on view copy, fix some tests * remove run_setup_anndata comments * fix transfer anndata test * fix test data format test * fix setup anndata test * fix most pyro tests * fix most scarches tests * remove state dict from field * Adapt all internal models to new setup (#1301) * adapt LDA * adapt linearscvi * remove _get_var_names_from_setup_anndata * adapt peakvi * adapt autozi * adapt scanvi * fix scanvi test * fix totalvi test * fix dataloader tests * fix multiple cov tests * adapt condscvi * adapt destvi * adapt multivi * fix setup compat test * remove get_from_registry util * fix scanvi and peakvi scarches tests * fix backwards compat tests and default missing summary stat in models * address comment * Adapt all external models to new setup (#1302) * adapt cellassign * adapt gimvi * adapt solo model * adapt stereoscope * codacy * rename _CONSTANTS to REGISTRY_KEYS (#1306) * Remove deprecated functions (#1308) * remove unused utils * remove references to removed functions in docs * Cleanup Manager register reference with AttrDict implementation (#1307) * implement attrdict and add it to properties of manager * use attrdict feature model code * address pr comments * codacy * view_anndata_setup new implementation (#1313) * implement attrdict and add it to properties of manager * use attrdict feature model code * address pr comments * view anndata setup implementation * fix bugs and tweak text * Field specific `view_anndata_setup` (#1315) * implement attrdict and add it to properties of manager * use attrdict feature model code * address pr comments * view anndata setup implementation * fix bugs and tweak text * field specific view setup * rich links in docstrings * update get_totalvi_protein_priors code w/ master * Fix docs (#1330) * codacy * address first round of comments * comments in constants file * add view registry tests * fix compat function * Update _layer_field.py * Update _obsm_field.py Co-authored-by: Adam Gayoso <adamgayoso@users.noreply.github.com>

* placeholder release notes for base * AnnData Manager refactor (scverse#1238) * move anndata file into anndata subfolder * mock out anndata manager and fields for categorical obs fields only * add layerfield * fix imports * fix imports in models * add temporary test and fix bugs in manager code * manager store on model class, refactor what needs to be defined for a model * clean up some existing code * restructure fields file, clean up more view anndata setup code, add more skeleton * add transfer_field for obs and layer fields * naive implementation of transfer_setup in anndata manager * implement backwards compat with setup dict on load * add docstrings to all new methods * address comments: change to registry_key, change fields to set in manager * add tests for transfer and manager from setup * remove anndata_fields method * address pr comment * Implement NonCategoricalJointObsField (scverse#1247) * implement continuous obsm field * make class specific to joint obs fields * make fields public * address comments * faster concat by subsetting obs * Implement CategoricalJointObsField (scverse#1248) * add categorical joint obs field * fix _make_obsm_categorical * add line in test * address pr comment * Pass manager to data loader (scverse#1280) * pass manager to data loader * address comment * Move and reorganize setup dict out of AnnData into Manager (scverse#1285) * new base field API for setup dict * adapt manager to new registry * backwards compatibility implementation * train works * adapt get_from_registry function * simplify needs transfer logic, working test_scvi * change _REGISTRY_KEYS back to _CONSTANTS * address comments, fix compat test * codacy * Add anndata registration classes to developer API (scverse#1292) * add anndata registration classes to developer API * move manager to anndata submodule to fix docs * Save/Load with saved setup inputs (scverse#1294) * skeleton for new save load, missing field logic * save load with saved setup inputs * add docstring for setup_inputs * codacy * fix documentation * additional pr comments * fix manager imports * Adapt TotalVI to new setup (scverse#1298) * obsm field * adapt totalVI to new setup * fix additional totalvi test * make field registration ordered * fix totalvi protein priors logic * address pr comments * address pr comments * Fix tests for SCVI, TOTALVI, generic AnnData utils (scverse#1299) * adapt totalVI to new setup * fix additional totalvi test * remove run_setup_anndata kwarg, reassign uuid on view copy, fix some tests * remove run_setup_anndata comments * fix transfer anndata test * fix test data format test * fix setup anndata test * fix most pyro tests * fix most scarches tests * remove state dict from field * Adapt all internal models to new setup (scverse#1301) * adapt LDA * adapt linearscvi * remove _get_var_names_from_setup_anndata * adapt peakvi * adapt autozi * adapt scanvi * fix scanvi test * fix totalvi test * fix dataloader tests * fix multiple cov tests * adapt condscvi * adapt destvi * adapt multivi * fix setup compat test * remove get_from_registry util * fix scanvi and peakvi scarches tests * fix backwards compat tests and default missing summary stat in models * address comment * Adapt all external models to new setup (scverse#1302) * adapt cellassign * adapt gimvi * adapt solo model * adapt stereoscope * codacy * rename _CONSTANTS to REGISTRY_KEYS (scverse#1306) * Remove deprecated functions (scverse#1308) * remove unused utils * remove references to removed functions in docs * Cleanup Manager register reference with AttrDict implementation (scverse#1307) * implement attrdict and add it to properties of manager * use attrdict feature model code * address pr comments * codacy * view_anndata_setup new implementation (scverse#1313) * implement attrdict and add it to properties of manager * use attrdict feature model code * address pr comments * view anndata setup implementation * fix bugs and tweak text * Field specific `view_anndata_setup` (scverse#1315) * implement attrdict and add it to properties of manager * use attrdict feature model code * address pr comments * view anndata setup implementation * fix bugs and tweak text * field specific view setup * rich links in docstrings * update get_totalvi_protein_priors code w/ master * Fix docs (scverse#1330) * codacy * address first round of comments * comments in constants file * add view registry tests * fix compat function * Update _layer_field.py * Update _obsm_field.py Co-authored-by: Adam Gayoso <adamgayoso@users.noreply.github.com>

justjhong added 15 commits January 11, 2022 17:12

adapt LDA

063a828

adapt linearscvi

188a22e

remove _get_var_names_from_setup_anndata

80d5605

adapt peakvi

2af6346

adapt autozi

fb23273

adapt scanvi

9f924c2

fix scanvi test

f05277c

fix totalvi test

cc0dfdb

fix dataloader tests

2b0eff8

fix multiple cov tests

8ca3649

adapt condscvi

7ceaf60

adapt destvi

342cf54

adapt multivi

ad93236

fix setup compat test

4c3e40a

remove get_from_registry util

d7217a9

justjhong requested a review from adamgayoso January 12, 2022 20:45

justjhong added 2 commits January 12, 2022 13:00

fix scanvi and peakvi scarches tests

e1056ba

fix backwards compat tests and default missing summary stat in models

60f54b5

adamgayoso reviewed Jan 14, 2022

View reviewed changes

justjhong added 2 commits January 14, 2022 10:44

address comment

14ec85e

Adapt all external models to new setup (#1302)

1878ac5

* adapt cellassign * adapt gimvi * adapt solo model * adapt stereoscope

justjhong merged commit c29fec8 into jhong/adatarefactor-testing Jan 14, 2022

justjhong deleted the jhong/adatarefactor-internalmodels branch January 14, 2022 18:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adapt all internal models to new setup #1301

Adapt all internal models to new setup #1301

justjhong commented Jan 12, 2022

adamgayoso Jan 14, 2022

justjhong Jan 14, 2022 •

edited

Loading

		An AnnDataField for labels which include explicitly unlabeled cells.

		Remaps unlabeled category to the final index if present in labels.

Adapt all internal models to new setup #1301

Adapt all internal models to new setup #1301

Conversation

justjhong commented Jan 12, 2022

adamgayoso Jan 14, 2022

Choose a reason for hiding this comment

justjhong Jan 14, 2022 • edited Loading

Choose a reason for hiding this comment

justjhong Jan 14, 2022 •

edited

Loading