Skip to content

chore: remove zappy and test zarr v3 #3884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Nov 10, 2025
Merged

chore: remove zappy and test zarr v3 #3884

merged 8 commits into from
Nov 10, 2025

Conversation

@flying-sheep
Copy link
Member

@flying-sheep flying-sheep commented Nov 7, 2025 •
edited
Loading

@flying-sheep flying-sheep added this to the 1.12.0 milestone Nov 7, 2025
@codecov
Copy link

codecov bot commented Nov 7, 2025 •
edited
Loading

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
2432 2 2430 137
View the full list of 2 ❄️ flaky test(s) 
tests/test_preprocessing_distributed.py::test_write_zarr[dask]

Flake rate in main: 3.57% (Passed 27 times, Failed 1 times)

Stack Traces | 0.559s run time 
adata = AnnData object with n_obs × n_vars = 10000 × 1000
    var: 'gene_ids'
    uns: 'log1p'
adata_dist = AnnData object with n_obs × n_vars = 10000 × 1000
    var: 'gene_ids'
    uns: 'dist-mode', 'log1p'
tmp_path = PosixPath('.../pytest-0/popen-gw0/test_write_zarr_dask_0')

    #x1B[0m#x1B[37m@pytest#x1B[39;49;00m.mark.filterwarnings(#x1B[33m"#x1B[39;49;00m#x1B[33mignore::anndata.OldFormatWarning#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m)#x1B[90m#x1B[39;49;00m
    #x1B[94mdef#x1B[39;49;00m#x1B[90m #x1B[39;49;00m#x1B[92mtest_write_zarr#x1B[39;49;00m(adata: AnnData, adata_dist: AnnData, tmp_path: Path) -> #x1B[94mNone#x1B[39;49;00m:#x1B[90m#x1B[39;49;00m
        log1p(adata_dist)#x1B[90m#x1B[39;49;00m
        #x1B[94massert#x1B[39;49;00m #x1B[96misinstance#x1B[39;49;00m(adata_dist.X, DIST_TYPES)#x1B[90m#x1B[39;49;00m
        chunks = adata_dist.X.chunks#x1B[90m#x1B[39;49;00m
        #x1B[94mif#x1B[39;49;00m #x1B[96misinstance#x1B[39;49;00m(chunks[#x1B[94m0#x1B[39;49;00m], #x1B[96mtuple#x1B[39;49;00m):#x1B[90m#x1B[39;49;00m
            chunks = (chunks[#x1B[94m0#x1B[39;49;00m][#x1B[94m0#x1B[39;49;00m],) + chunks[#x1B[94m1#x1B[39;49;00m]#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        #x1B[90m# write metadata using regular anndata#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
        root = tmp_path / #x1B[33m"#x1B[39;49;00m#x1B[33mtest.zarr#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
        adata.write_zarr(root, chunks=chunks)#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        #x1B[90m# overwrite X#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
        #x1B[94mif#x1B[39;49;00m adata_dist.uns[#x1B[33m"#x1B[39;49;00m#x1B[33mdist-mode#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m] == #x1B[33m"#x1B[39;49;00m#x1B[33mdask#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m:#x1B[90m#x1B[39;49;00m
            adata_dist.X.to_zarr(root / #x1B[33m"#x1B[39;49;00m#x1B[33mX#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m, overwrite=#x1B[94mTrue#x1B[39;49;00m)#x1B[90m#x1B[39;49;00m
        #x1B[94melif#x1B[39;49;00m adata_dist.uns[#x1B[33m"#x1B[39;49;00m#x1B[33mdist-mode#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m] == #x1B[33m"#x1B[39;49;00m#x1B[33mdirect#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m:#x1B[90m#x1B[39;49;00m
            adata_dist.X.to_zarr(root / #x1B[33m"#x1B[39;49;00m#x1B[33mX#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m, chunks=chunks)#x1B[90m#x1B[39;49;00m
        #x1B[94melse#x1B[39;49;00m:#x1B[90m#x1B[39;49;00m
            pytest.fail(#x1B[33m"#x1B[39;49;00m#x1B[33madd branch for new dist-mode#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m)#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        #x1B[90m# read back as zarr directly and check it is the same as adata.X#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
        adata_log1p = read_zarr(root)#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        log1p(adata)#x1B[90m#x1B[39;49;00m
>       npt.assert_allclose(adata_log1p.X, adata.X)#x1B[90m#x1B[39;49;00m
#x1B[1m#x1B[31mE       AssertionError: #x1B[0m
#x1B[1m#x1B[31mE       Not equal to tolerance rtol=1e-07, atol=0#x1B[0m
#x1B[1m#x1B[31mE       #x1B[0m
#x1B[1m#x1B[31mE       Mismatched elements: 691914 / 10000000 (6.92%)#x1B[0m
#x1B[1m#x1B[31mE       Max absolute difference among violations: 6.437752#x1B[0m
#x1B[1m#x1B[31mE       Max relative difference among violations: 1.#x1B[0m
#x1B[1m#x1B[31mE        ACTUAL: array([[0., 0., 0., ..., 0., 0., 0.],#x1B[0m
#x1B[1m#x1B[31mE              [0., 0., 0., ..., 0., 0., 0.],#x1B[0m
#x1B[1m#x1B[31mE              [0., 0., 0., ..., 0., 0., 0.],...#x1B[0m
#x1B[1m#x1B[31mE        DESIRED: array([[0., 0., 0., ..., 0., 0., 0.],#x1B[0m
#x1B[1m#x1B[31mE              [0., 0., 0., ..., 0., 0., 0.],#x1B[0m
#x1B[1m#x1B[31mE              [0., 0., 0., ..., 0., 0., 0.],...#x1B[0m

#x1B[1m#x1B[31mtests/test_preprocessing_distributed.py#x1B[0m:171: AssertionError
tests/test_preprocessing_distributed.py::test_write_zarr[direct]

Flake rate in main: 3.57% (Passed 27 times, Failed 1 times)

Stack Traces | 0.659s run time 
adata = AnnData object with n_obs × n_vars = 10000 × 1000
    var: 'gene_ids'
    uns: 'log1p'
adata_dist = AnnData object with n_obs × n_vars = 10000 × 1000
    var: 'gene_ids'
    uns: 'dist-mode', 'log1p'
tmp_path = PosixPath('.../pytest-0/popen-gw0/test_write_zarr_direct_0')

    #x1B[0m#x1B[37m@pytest#x1B[39;49;00m.mark.filterwarnings(#x1B[33m"#x1B[39;49;00m#x1B[33mignore::anndata.OldFormatWarning#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m)#x1B[90m#x1B[39;49;00m
    #x1B[94mdef#x1B[39;49;00m#x1B[90m #x1B[39;49;00m#x1B[92mtest_write_zarr#x1B[39;49;00m(adata: AnnData, adata_dist: AnnData, tmp_path: Path) -> #x1B[94mNone#x1B[39;49;00m:#x1B[90m#x1B[39;49;00m
        log1p(adata_dist)#x1B[90m#x1B[39;49;00m
        #x1B[94massert#x1B[39;49;00m #x1B[96misinstance#x1B[39;49;00m(adata_dist.X, DIST_TYPES)#x1B[90m#x1B[39;49;00m
        chunks = adata_dist.X.chunks#x1B[90m#x1B[39;49;00m
        #x1B[94mif#x1B[39;49;00m #x1B[96misinstance#x1B[39;49;00m(chunks[#x1B[94m0#x1B[39;49;00m], #x1B[96mtuple#x1B[39;49;00m):#x1B[90m#x1B[39;49;00m
            chunks = (chunks[#x1B[94m0#x1B[39;49;00m][#x1B[94m0#x1B[39;49;00m],) + chunks[#x1B[94m1#x1B[39;49;00m]#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        #x1B[90m# write metadata using regular anndata#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
        root = tmp_path / #x1B[33m"#x1B[39;49;00m#x1B[33mtest.zarr#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
        adata.write_zarr(root, chunks=chunks)#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        #x1B[90m# overwrite X#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
        #x1B[94mif#x1B[39;49;00m adata_dist.uns[#x1B[33m"#x1B[39;49;00m#x1B[33mdist-mode#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m] == #x1B[33m"#x1B[39;49;00m#x1B[33mdask#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m:#x1B[90m#x1B[39;49;00m
            adata_dist.X.to_zarr(root / #x1B[33m"#x1B[39;49;00m#x1B[33mX#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m, overwrite=#x1B[94mTrue#x1B[39;49;00m)#x1B[90m#x1B[39;49;00m
        #x1B[94melif#x1B[39;49;00m adata_dist.uns[#x1B[33m"#x1B[39;49;00m#x1B[33mdist-mode#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m] == #x1B[33m"#x1B[39;49;00m#x1B[33mdirect#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m:#x1B[90m#x1B[39;49;00m
            adata_dist.X.to_zarr(root / #x1B[33m"#x1B[39;49;00m#x1B[33mX#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m, chunks=chunks)#x1B[90m#x1B[39;49;00m
        #x1B[94melse#x1B[39;49;00m:#x1B[90m#x1B[39;49;00m
            pytest.fail(#x1B[33m"#x1B[39;49;00m#x1B[33madd branch for new dist-mode#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m)#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        #x1B[90m# read back as zarr directly and check it is the same as adata.X#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
        adata_log1p = read_zarr(root)#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        log1p(adata)#x1B[90m#x1B[39;49;00m
>       npt.assert_allclose(adata_log1p.X, adata.X)#x1B[90m#x1B[39;49;00m
#x1B[1m#x1B[31mE       AssertionError: #x1B[0m
#x1B[1m#x1B[31mE       Not equal to tolerance rtol=1e-07, atol=0#x1B[0m
#x1B[1m#x1B[31mE       #x1B[0m
#x1B[1m#x1B[31mE       Mismatched elements: 691914 / 10000000 (6.92%)#x1B[0m
#x1B[1m#x1B[31mE       Max absolute difference among violations: 6.437752#x1B[0m
#x1B[1m#x1B[31mE       Max relative difference among violations: 1.#x1B[0m
#x1B[1m#x1B[31mE        ACTUAL: array([[0., 0., 0., ..., 0., 0., 0.],#x1B[0m
#x1B[1m#x1B[31mE              [0., 0., 0., ..., 0., 0., 0.],#x1B[0m
#x1B[1m#x1B[31mE              [0., 0., 0., ..., 0., 0., 0.],...#x1B[0m
#x1B[1m#x1B[31mE        DESIRED: array([[0., 0., 0., ..., 0., 0., 0.],#x1B[0m
#x1B[1m#x1B[31mE              [0., 0., 0., ..., 0., 0., 0.],#x1B[0m
#x1B[1m#x1B[31mE              [0., 0., 0., ..., 0., 0., 0.],...#x1B[0m

#x1B[1m#x1B[31mtests/test_preprocessing_distributed.py#x1B[0m:171: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@flying-sheep flying-sheep changed the title chore: test zarr v3 ci: test zarr v3 Nov 10, 2025
@flying-sheep flying-sheep marked this pull request as ready for review November 10, 2025 10:15
ilan-gold
ilan-gold approved these changes Nov 10, 2025
View reviewed changes
Copy link
Contributor

@ilan-gold ilan-gold left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Release note for removing zappy

@flying-sheep flying-sheep changed the title ci: test zarr v3 chore: remove zappy and test zarr v3 Nov 10, 2025
@flying-sheep flying-sheep merged commit 819ca8c into main Nov 10, 2025
16 checks passed
@flying-sheep flying-sheep deleted the pa/unpin-zarr branch November 10, 2025 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ilan-gold ilan-gold ilan-gold approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

1.12.0

Development

Successfully merging this pull request may close these issues.

Remove zappy

3 participants

@flying-sheep @ilan-gold