fix(dataset): harden dataset IO edge cases#2410
Merged
Merged
Conversation
- Avoid mutating caller-owned Detections during dataset construction and reject invalid class ids with clear ValueErrors. - Make COCO loading/export tolerant of missing optional metadata, add from_coco(use_iscrowd), and export mask pixel area when needed. - Let folder-structure and YOLO loading skip common clutter and accept PIL-readable image modes with regression coverage. --- Co-authored-by: Codex <codex@openai.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #2410 +/- ##
=======================================
Coverage 86% 87%
=======================================
Files 70 70
Lines 10175 10198 +23
=======================================
+ Hits 8799 8823 +24
+ Misses 1376 1375 -1 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR hardens supervision.dataset import/export paths (COCO + YOLO) and dataset construction to better handle real-world edge cases, while expanding regression coverage across these scenarios.
Changes:
DetectionDatasetnow avoids mutating caller-ownedDetectionsby copying annotations, and validatesclass_iddtype/range early with clearerValueErrors.- COCO IO is more tolerant/controllable: missing optional
iscrowd/areafields default safely,DetectionDataset.from_coco(use_iscrowd=...)is added, and COCO export prefers mask pixel area when no storedareaexists. - YOLO IO now reads only image dimensions via PIL (context-managed) and accepts PIL-readable image modes; classification folder-structure loading is more robust to dotfiles/root clutter.
Review scores (per CONTRIBUTING guidance):
- Code quality: 4/5
- Testing: 5/5
- Docs/Changelog: 4/5
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
src/supervision/dataset/core.py |
Copies/validates detection annotations in constructor; adds use_iscrowd plumbing; hardens folder-structure loading. |
src/supervision/dataset/formats/coco.py |
Defaults missing iscrowd/area; exports mask pixel area when area missing. |
src/supervision/dataset/formats/yolo.py |
Uses context-managed PIL open for dimensions; removes restrictive mode validation. |
tests/dataset/test_core.py |
Adds tests for non-mutation, class id validation, and root clutter handling. |
tests/dataset/formats/test_coco.py |
Adds tests for missing COCO metadata defaults, use_iscrowd, and mask-area export behavior. |
tests/dataset/formats/test_yolo.py |
Adds test for non-RGB (PIL-readable) image mode acceptance. |
docs/changelog.md |
Documents the dataset IO/export edge-case fixes. |
- Preserve from_coco positional show_progress compatibility while keeping use_iscrowd keyword-only. - Filter class-folder loading to image files and export missing COCO mask area from decoded masks. - Add regression coverage, changelog updates, and types-tqdm for mypy. --- Co-authored-by: Codex <codex@openai.com>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request makes several improvements and bug fixes to dataset IO and export logic, especially for COCO and YOLO formats, and strengthens validation and safety in the
DetectionDatasetconstructor. It also improves robustness when loading datasets from folder structures and adds comprehensive test coverage for these cases.Dataset IO and Validation Improvements:
DetectionDatasetconstructor now deep-copies input annotation objects to avoid mutating caller-ownedDetections, and validates that allclass_idvalues are within the valid range for the providedclasses, raising a clearValueErrorotherwise. [1] [2] [3]COCO Format Handling:
iscrowdandareafields, and exposes ause_iscrowdparameter inDetectionDataset.from_cocoto control whether these fields are included inDetections.data. [1] [2] [3] [4] [5]YOLO Format Handling:
Changelog and Documentation:
Test Coverage:
These changes make dataset loading and export safer, more robust, and more standards-compliant, while providing clearer error messages and better test coverage.
part of #2408