Skip to content

fix(dataset): harden dataset IO edge cases#2410

Merged
Borda merged 2 commits into
developfrom
medium/Dataset-1
Jul 7, 2026
Merged

fix(dataset): harden dataset IO edge cases#2410
Borda merged 2 commits into
developfrom
medium/Dataset-1

Conversation

@Borda

@Borda Borda commented Jul 7, 2026

Copy link
Copy Markdown
Member

This pull request makes several improvements and bug fixes to dataset IO and export logic, especially for COCO and YOLO formats, and strengthens validation and safety in the DetectionDataset constructor. It also improves robustness when loading datasets from folder structures and adds comprehensive test coverage for these cases.

Dataset IO and Validation Improvements:

  • The DetectionDataset constructor now deep-copies input annotation objects to avoid mutating caller-owned Detections, and validates that all class_id values are within the valid range for the provided classes, raising a clear ValueError otherwise. [1] [2] [3]
  • Loading from a folder structure now ignores dotfiles, loose files, and nested dotfiles, preventing root clutter from breaking dataset loading. [1] [2]

COCO Format Handling:

  • The COCO loader now accepts and defaults missing optional iscrowd and area fields, and exposes a use_iscrowd parameter in DetectionDataset.from_coco to control whether these fields are included in Detections.data. [1] [2] [3] [4] [5]
  • When exporting to COCO, if no area is present in the data but a mask is available, the pixel area of the mask is used instead of the bounding box area. [1] [2]

YOLO Format Handling:

  • YOLO loader now accepts any image readable by PIL (such as RGBA or palette PNGs), not just RGB or grayscale, since only dimensions are required. [1] [2]

Changelog and Documentation:

  • The changelog documents all these fixes, including clearer error handling, non-mutating dataset construction, improved COCO/YOLO edge case support, and folder structure robustness.

Test Coverage:

  • New tests cover the non-mutation guarantee, class id validation, COCO metadata handling, mask area export, YOLO image mode acceptance, and folder structure clutter handling. [1] [2] [3] [4] [5]

These changes make dataset loading and export safer, more robust, and more standards-compliant, while providing clearer error messages and better test coverage.


part of #2408

- Avoid mutating caller-owned Detections during dataset construction and reject invalid class ids with clear ValueErrors.
- Make COCO loading/export tolerant of missing optional metadata, add from_coco(use_iscrowd), and export mask pixel area when needed.
- Let folder-structure and YOLO loading skip common clutter and accept PIL-readable image modes with regression coverage.

---

Co-authored-by: Codex <codex@openai.com>
@Borda Borda requested a review from Copilot July 7, 2026 10:53
@Borda Borda requested a review from SkalskiP as a code owner July 7, 2026 10:53
@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87%. Comparing base (5b4c8b6) to head (d68f81f).

Additional details and impacted files
@@           Coverage Diff           @@
##           develop   #2410   +/-   ##
=======================================
  Coverage       86%     87%           
=======================================
  Files           70      70           
  Lines        10175   10198   +23     
=======================================
+ Hits          8799    8823   +24     
+ Misses        1376    1375    -1     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens supervision.dataset import/export paths (COCO + YOLO) and dataset construction to better handle real-world edge cases, while expanding regression coverage across these scenarios.

Changes:

  • DetectionDataset now avoids mutating caller-owned Detections by copying annotations, and validates class_id dtype/range early with clearer ValueErrors.
  • COCO IO is more tolerant/controllable: missing optional iscrowd/area fields default safely, DetectionDataset.from_coco(use_iscrowd=...) is added, and COCO export prefers mask pixel area when no stored area exists.
  • YOLO IO now reads only image dimensions via PIL (context-managed) and accepts PIL-readable image modes; classification folder-structure loading is more robust to dotfiles/root clutter.

Review scores (per CONTRIBUTING guidance):

  • Code quality: 4/5
  • Testing: 5/5
  • Docs/Changelog: 4/5

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/supervision/dataset/core.py Copies/validates detection annotations in constructor; adds use_iscrowd plumbing; hardens folder-structure loading.
src/supervision/dataset/formats/coco.py Defaults missing iscrowd/area; exports mask pixel area when area missing.
src/supervision/dataset/formats/yolo.py Uses context-managed PIL open for dimensions; removes restrictive mode validation.
tests/dataset/test_core.py Adds tests for non-mutation, class id validation, and root clutter handling.
tests/dataset/formats/test_coco.py Adds tests for missing COCO metadata defaults, use_iscrowd, and mask-area export behavior.
tests/dataset/formats/test_yolo.py Adds test for non-RGB (PIL-readable) image mode acceptance.
docs/changelog.md Documents the dataset IO/export edge-case fixes.

Comment thread src/supervision/dataset/core.py
- Preserve from_coco positional show_progress compatibility while keeping use_iscrowd keyword-only.
- Filter class-folder loading to image files and export missing COCO mask area from decoded masks.
- Add regression coverage, changelog updates, and types-tqdm for mypy.

---

Co-authored-by: Codex <codex@openai.com>
@socket-security

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedtypes-tqdm@​4.68.0.20260608100100100100100

View full report

@Borda Borda merged commit 6a69197 into develop Jul 7, 2026
26 checks passed
@Borda Borda deleted the medium/Dataset-1 branch July 7, 2026 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants