Leaner CI matrix + test_io deadlock

[ClaudiaComito]

Due Diligence

• General:
  • title of the PR is suitable to appear in the Release Notes
• Implementation:
  • unit tests: all split configurations tested
  • unit tests: multiple dtypes tested
  • NEW unit tests: MPS tested (1 MPI process, 1 GPU)
  • benchmarks: created for new functionality
  • benchmarks: performance improved or maintained
  • documentation updated where needed

Description

This PR implements a leaner CI matrix as discussed in #2134. The race condition + deadlock arising in test_io with Python 3.14 has also been solved.

Issue/s resolved: #2134

Changes proposed:

• refactor ci.yaml as per Prune CI matrix #2134 . Runs for every commit as requested by @brownbaerchen
• keep the full matrix in ci_full.yaml. This will run when a PR is merged into main, if the PR affects the codebase
• in test_io.test_load_zarr_group prevent race condition by inserting a Barrier before starting to write out the zarr group. Might be related to PEP 703 in combination with Zarr V3's asynchronous I/O.

Type of change

• Bug fix (non-breaking change which fixes an issue) + GH action refactoring

Memory requirements

DNA

Performance

DNA

Does this change modify the behaviour of other functions? If so, which?

no

[github-actions]

Thank you for the PR!

[github-actions]

Thank you for the PR!

[github-actions]

Thank you for the PR!

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.63%. Comparing base (a6109ec) to head (583c941).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2139   +/-   ##
=======================================
  Coverage   91.63%   91.63%           
=======================================
  Files          89       89           
  Lines       14003    14003           
=======================================
  Hits        12832    12832           
  Misses       1171     1171           

Flag	Coverage Δ
unit	91.63% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

Thank you for the PR!

[github-actions]

Thank you for the PR!

[github-actions]

Thank you for the PR!

[brownbaerchen]

Can anybody reproduce the deadlock locally? I can't with Python 3.14 and torch==2.9.1 torchvision==0.24.1 torchaudio==2.9.1.

[JuanPedroGHM]

Can anybody reproduce the deadlock locally? I can't with Python 3.14 and torch==2.9.1 torchvision==0.24.1 torchaudio==2.9.1.

I could not reproduce it. Tried with torch==2.10.

[brownbaerchen]

@copilot, implement a time limit of 2 hours for the new lean CI and the full CI. Also, fix the deadlock in test_io. Do both by suggesting changes on this PR.

[Copilot]

@brownbaerchen I've opened a new pull request, #2147, to work on those changes. Once the pull request is ready, I'll request review from you.

[github-actions]

Thank you for the PR!

[github-actions]

Thank you for the PR!

[brownbaerchen]

Adding pandas back in as optional dependency (needs #2149)

[github-actions]

Thank you for the PR!

Not relevant, addressed in pr #2149