feat: Auto-detect parallel mode, fix overlap-based RAM estimation #9
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Auto-detect parallel mode: enabled for multi-file formats (individual TIFFs, zarr), disabled for single-file OME-TIFF (benchmarks showed 40% slower due to I/O contention) - Fix RAM estimation to use actual overlap region size instead of full tile size (was 5-10x over-conservative) - Add hybrid batching: simple path when memory fits, batched when >30% RAM needed - Use class max_workers setting instead of hardcoded values - Add worker count logging for transparency - Add tests for overlap bounds calculation 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimizes parallel tile registration by introducing smart RAM management and format-based auto-detection. The key improvement is fixing the RAM estimation to use actual overlap region sizes instead of full tile sizes, which reduces memory requirements by 5-10x. The PR adds a hybrid batching approach that switches to batched processing only when memory usage exceeds 30% of available RAM, and auto-detects whether to enable parallel processing based on file format to avoid I/O contention in single-file OME-TIFF.
- Auto-detect parallel mode based on file format (disabled for single-file OME-TIFF due to I/O contention)
- Fix RAM estimation to calculate actual overlap region size instead of full tile size
- Add hybrid batching that only activates when estimated memory exceeds 30% of RAM
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| tests/test_registration.py | Adds new test class TestOverlapBoundsSize to verify that overlap regions are correctly calculated and much smaller than full tiles |
| src/tilefusion/core.py | Implements auto-detection for parallel mode, fixes RAM estimation to use actual overlap bounds, adds hybrid batching logic, and uses configurable _max_workers throughout |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
hongquanli
changed the title
hongquanli
added a commit
that referenced
this pull request
Jan 4, 2026
- Fix #9: Preserve flatfield data when checkbox is toggled off (just hide UI) - Fix #10: Combine error message string into single line - Fix #11: Add shape validation in apply_flatfield and apply_flatfield_region - Fix #12: Add error handling in load_flatfield for invalid file formats 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
max_workersinstead of hardcoded valuesBenchmark Results
Test Datasets
Registration Performance (8 workers)
OME-TIFF is slower with parallel due to single-file I/O contention → auto-disabled
Test plan
pytest tests/test_registration.py- all 13 tests pass🤖 Generated with Claude Code