Skip to content

fix: PCA centering in ingest#4208

Merged
flying-sheep merged 4 commits into
mainfrom
fix-ingest-PCA
Jul 21, 2026
Merged

fix: PCA centering in ingest#4208
flying-sheep merged 4 commits into
mainfrom
fix-ingest-PCA

Conversation

@Intron7

@Intron7 Intron7 commented Jul 13, 2026

Copy link
Copy Markdown
Member

Ingest centers its PCA against the query data not the fit data. I also made PCA sparse

@Intron7
fix PCA centering
5f4a407 
Signed-off-by: Intron7 <severin.dicks@icloud.com>
@codecov

codecov Bot commented Jul 13, 2026 •
edited
Loading

Copy link
Copy Markdown

❌ 3 Tests Failed:

Tests completed Failed Passed Skipped
2970 3 2967 165
View the top 3 failed test(s) by shortest run time 
tests/test_plotting_embedded/test_embeddings.py::test_missing_values_categorical[spatial-na_color.black_tup-na_in_legend.True-legend.on_bottom-groups.all]
Stack Traces | 0.631s run time 
request = <FixtureRequest for <Function test_missing_values_categorical[spatial-na_color.black_tup-na_in_legend.True-legend.on_bottom-groups.all]>>
image_comparer = <function image_comparer.<locals>.save_and_compare at 0x7f090c072e50>
adata = AnnData object with n_obs × n_vars = 2600 × 20
    obs: 'label', 'label_missing', '1_missing'
    uns: 'spatial', 'pca', 'label_colors', 'label_missing_colors'
    obsm: 'spatial', 'X_pca'
    varm: 'PCs'
    layers: None (.X)
plotfunc = functools.partial(<function spatial at 0x7f08eade7950>, show=False)
na_color = (0, 0, 0, 1), na_in_legend = True, legend_loc = 'lower center'
groupsfunc = <function <lambda> at 0x7f0915262fb0>

    #x1B[0m#x1B[94mdef#x1B[39;49;00m#x1B[90m #x1B[39;49;00m#x1B[92mtest_missing_values_categorical#x1B[39;49;00m(#x1B[90m#x1B[39;49;00m
        *,#x1B[90m#x1B[39;49;00m
        request: pytest.FixtureRequest,#x1B[90m#x1B[39;49;00m
        image_comparer,#x1B[90m#x1B[39;49;00m
        adata,#x1B[90m#x1B[39;49;00m
        plotfunc,#x1B[90m#x1B[39;49;00m
        na_color,#x1B[90m#x1B[39;49;00m
        na_in_legend,#x1B[90m#x1B[39;49;00m
        legend_loc,#x1B[90m#x1B[39;49;00m
        groupsfunc,#x1B[90m#x1B[39;49;00m
    ):#x1B[90m#x1B[39;49;00m
        save_and_compare_images = partial(image_comparer, MISSING_VALUES_ROOT, tol=#x1B[94m15#x1B[39;49;00m)#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        base_name = request.node.name#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        #x1B[90m# Passing through a dict so it's easier to use default values#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
        kwargs = {}#x1B[90m#x1B[39;49;00m
        kwargs[#x1B[33m"#x1B[39;49;00m#x1B[33mlegend_loc#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m] = legend_loc#x1B[90m#x1B[39;49;00m
        kwargs[#x1B[33m"#x1B[39;49;00m#x1B[33mgroups#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m] = groupsfunc(adata.obs[#x1B[33m"#x1B[39;49;00m#x1B[33mlabel#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m])#x1B[90m#x1B[39;49;00m
        #x1B[94mif#x1B[39;49;00m na_color #x1B[95mis#x1B[39;49;00m #x1B[95mnot#x1B[39;49;00m #x1B[94mNone#x1B[39;49;00m:#x1B[90m#x1B[39;49;00m
            kwargs[#x1B[33m"#x1B[39;49;00m#x1B[33mna_color#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m] = na_color#x1B[90m#x1B[39;49;00m
        kwargs[#x1B[33m"#x1B[39;49;00m#x1B[33mna_in_legend#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m] = na_in_legend#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        plotfunc(adata, color=[#x1B[33m"#x1B[39;49;00m#x1B[33mlabel#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m, #x1B[33m"#x1B[39;49;00m#x1B[33mlabel_missing#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m], **kwargs)#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
>       save_and_compare_images(base_name)#x1B[90m#x1B[39;49;00m
#x1B[1m#x1B[31mE       AssertionError: Image files did not match.#x1B[0m
#x1B[1m#x1B[31mE       RMS Value:  15.827322403052634#x1B[0m
#x1B[1m#x1B[31mE       Expected:   file:.../tests/_images/3.11/embedding-missing-values/test_missing_values_categorical%5Bspatial-na_color.black_tup-na_in_legend.True-legend.on_bottom-groups.all%5D/expected.png#x1B[0m
#x1B[1m#x1B[31mE       Actual:     file:.../tests/_images/3.11/embedding-missing-values/test_missing_values_categorical%5Bspatial-na_color.black_tup-na_in_legend.True-legend.on_bottom-groups.all%5D/actual.png#x1B[0m
#x1B[1m#x1B[31mE       Difference: file:.../tests/_images/3.11/embedding-missing-values/test_missing_values_categorical%5Bspatial-na_color.black_tup-na_in_legend.True-legend.on_bottom-groups.all%5D/actual-failed-diff.png#x1B[0m
#x1B[1m#x1B[31mE       Tolerance:  15#x1B[0m

#x1B[1m#x1B[31mtests/test_plotting_embedded/test_embeddings.py#x1B[0m:129: AssertionError
tests/test_plotting_embedded/test_embeddings.py::test_missing_values_categorical[spatial-na_color.black_tup-na_in_legend.False-legend.on_bottom-groups.all]
Stack Traces | 0.866s run time 
request = <FixtureRequest for <Function test_missing_values_categorical[spatial-na_color.black_tup-na_in_legend.False-legend.on_bottom-groups.all]>>
image_comparer = <function image_comparer.<locals>.save_and_compare at 0x7f08fcb33e20>
adata = AnnData object with n_obs × n_vars = 2600 × 20
    obs: 'label', 'label_missing', '1_missing'
    uns: 'spatial', 'pca', 'label_colors', 'label_missing_colors'
    obsm: 'spatial', 'X_pca'
    varm: 'PCs'
    layers: None (.X)
plotfunc = functools.partial(<function spatial at 0x7f08fcb31b10>, show=False)
na_color = (0, 0, 0, 1), na_in_legend = False, legend_loc = 'lower center'
groupsfunc = <function <lambda> at 0x7f0915262fb0>

    #x1B[0m#x1B[94mdef#x1B[39;49;00m#x1B[90m #x1B[39;49;00m#x1B[92mtest_missing_values_categorical#x1B[39;49;00m(#x1B[90m#x1B[39;49;00m
        *,#x1B[90m#x1B[39;49;00m
        request: pytest.FixtureRequest,#x1B[90m#x1B[39;49;00m
        image_comparer,#x1B[90m#x1B[39;49;00m
        adata,#x1B[90m#x1B[39;49;00m
        plotfunc,#x1B[90m#x1B[39;49;00m
        na_color,#x1B[90m#x1B[39;49;00m
        na_in_legend,#x1B[90m#x1B[39;49;00m
        legend_loc,#x1B[90m#x1B[39;49;00m
        groupsfunc,#x1B[90m#x1B[39;49;00m
    ):#x1B[90m#x1B[39;49;00m
        save_and_compare_images = partial(image_comparer, MISSING_VALUES_ROOT, tol=#x1B[94m15#x1B[39;49;00m)#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        base_name = request.node.name#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        #x1B[90m# Passing through a dict so it's easier to use default values#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
        kwargs = {}#x1B[90m#x1B[39;49;00m
        kwargs[#x1B[33m"#x1B[39;49;00m#x1B[33mlegend_loc#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m] = legend_loc#x1B[90m#x1B[39;49;00m
        kwargs[#x1B[33m"#x1B[39;49;00m#x1B[33mgroups#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m] = groupsfunc(adata.obs[#x1B[33m"#x1B[39;49;00m#x1B[33mlabel#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m])#x1B[90m#x1B[39;49;00m
        #x1B[94mif#x1B[39;49;00m na_color #x1B[95mis#x1B[39;49;00m #x1B[95mnot#x1B[39;49;00m #x1B[94mNone#x1B[39;49;00m:#x1B[90m#x1B[39;49;00m
            kwargs[#x1B[33m"#x1B[39;49;00m#x1B[33mna_color#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m] = na_color#x1B[90m#x1B[39;49;00m
        kwargs[#x1B[33m"#x1B[39;49;00m#x1B[33mna_in_legend#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m] = na_in_legend#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        plotfunc(adata, color=[#x1B[33m"#x1B[39;49;00m#x1B[33mlabel#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m, #x1B[33m"#x1B[39;49;00m#x1B[33mlabel_missing#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m], **kwargs)#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
>       save_and_compare_images(base_name)#x1B[90m#x1B[39;49;00m
#x1B[1m#x1B[31mE       AssertionError: Image files did not match.#x1B[0m
#x1B[1m#x1B[31mE       RMS Value:  15.780919928103211#x1B[0m
#x1B[1m#x1B[31mE       Expected:   file:.../tests/_images/3.11/embedding-missing-values/test_missing_values_categorical%5Bspatial-na_color.black_tup-na_in_legend.False-legend.on_bottom-groups.all%5D/expected.png#x1B[0m
#x1B[1m#x1B[31mE       Actual:     file:.../tests/_images/3.11/embedding-missing-values/test_missing_values_categorical%5Bspatial-na_color.black_tup-na_in_legend.False-legend.on_bottom-groups.all%5D/actual.png#x1B[0m
#x1B[1m#x1B[31mE       Difference: file:.../tests/_images/3.11/embedding-missing-values/test_missing_values_categorical%5Bspatial-na_color.black_tup-na_in_legend.False-legend.on_bottom-groups.all%5D/actual-failed-diff.png#x1B[0m
#x1B[1m#x1B[31mE       Tolerance:  15#x1B[0m

#x1B[1m#x1B[31mtests/test_plotting_embedded/test_embeddings.py#x1B[0m:129: AssertionError
tests/notebooks/test_pbmc3k.py::test_pbmc3k
Stack Traces | 13.4s run time 
contains 2 failed subtests

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@Intron7
add release note
2d4cffc 
Signed-off-by: Intron7 <severin.dicks@icloud.com>
@Intron7
Intron7 requested a review from flying-sheep July 13, 2026 13:09
flying-sheep
flying-sheep reviewed Jul 13, 2026
View reviewed changes
Comment thread tests/test_ingest.py
def fail_toarray(*args, **kwargs) -> None:
pytest.fail("PCA ingest must not densify the query matrix")

monkeypatch.setattr(type(adata_new.X), "toarray", fail_toarray)

@flying-sheep flying-sheep Jul 13, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a bit fragile, but I can’t think of a better way …

@flying-sheep

flying-sheep commented Jul 13, 2026

Copy link
Copy Markdown
Member

just add a relnote

@flying-sheep flying-sheep changed the title fix PCA centering in ingest fix: PCA centering in ingest Jul 13, 2026
@Intron7

Intron7 commented Jul 21, 2026

Copy link
Copy Markdown
Member Author

@flying-sheep already done.

@Intron7 Intron7 added this to the 1.12.3 milestone Jul 21, 2026
@flying-sheep

flying-sheep commented Jul 21, 2026

Copy link
Copy Markdown
Member

huh strange, I wouldn’t have said that If I had seen it. Sorry!

@flying-sheep
flying-sheep enabled auto-merge (squash) July 21, 2026 12:40
@flying-sheep
flying-sheep disabled auto-merge July 21, 2026 14:15
@flying-sheep
flying-sheep merged commit 2f52f59 into main Jul 21, 2026
9 of 12 checks passed
@flying-sheep
flying-sheep deleted the fix-ingest-PCA branch July 21, 2026 14:15
@lumberbot-app

lumberbot-app Bot commented Jul 21, 2026

Copy link
Copy Markdown

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
git checkout 1.12.x
git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
git cherry-pick -x -m1 2f52f59a95ca5b37f268b5a569ff2520f35a1cc6
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
git commit -am 'Backport PR #4208: fix: PCA centering in ingest'
  1. Push to a named branch:
git push YOURFORK 1.12.x:auto-backport-of-pr-4208-on-1.12.x
  1. Create a PR against branch 1.12.x, I would have named this PR:

"Backport PR #4208 on branch 1.12.x (fix: PCA centering in ingest)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

@flying-sheep flying-sheep mentioned this pull request Jul 23, 2026
Merged
flying-sheep added a commit that referenced this pull request Jul 23, 2026
@flying-sheep @Intron7
Backport PR #4208 on branch 1.12.x (fix: PCA centering in ingest) (#4235
e4a546e 
)

Co-authored-by: Severin Dicks <37635888+Intron7@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@flying-sheep flying-sheep flying-sheep approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

1.12.3

Development

Successfully merging this pull request may close these issues.

2 participants

@Intron7 @flying-sheep