Skip to content

Support for IOS Matching Metric. Introduced the mask_non_max_merge function for handling non-maximum merging of masks #1774

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: develop
Choose a base branch
from

Conversation

SunHao-AI
Copy link

Description

When I used the yolo11-seg model for slice inference, I found that the merging did not work well. After analysis, it is found that for slender objects, its mask area only accounts for a small part, and the box area may be very large, so it is necessary to use mask for iou calculation. aaa
When I implemented the mask_non_max_merge function, I found that the merge effect was still not satisfactory, as shown in the figure:
NMM-IOU10
I found that the sahi library implemented the non-maximum merge algorithm of the IOS version, so I made changes to the code.
NMM-IOS10

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

import cv2
import numpy as np
from PIL import Image
from ultralytics import YOLO

import supervision as sv

model = YOLO("yolon11-seg.pt")
image = cv2.imread(<your_image_path>)


def callback(image_slice: np.ndarray) -> sv.Detections:
    results = model(image_slice)[0]
    return sv.Detections.from_ultralytics(results)

# slicer = sv.InferenceSlicer(callback=callback, slice_wh=(960, 960), overlap_ratio_wh=None, overlap_wh=(50, 50), overlap_filter="non_max_merge", iou_threshold=0.1, match_metric="IOU")
slicer = sv.InferenceSlicer(callback=callback, slice_wh=(960, 960), overlap_ratio_wh=None, overlap_wh=(50, 50), overlap_filter="non_max_merge", iou_threshold=0.1, match_metric="IOS")
# slicer = sv.InferenceSlicer(callback=callback, slice_wh=(960, 960), overlap_ratio_wh=None, overlap_wh=(50, 50), overlap_filter="non_max_suppression", iou_threshold=0.1, match_metric="IOU")
# slicer = sv.InferenceSlicer(callback=callback, slice_wh=(960, 960), overlap_ratio_wh=None, overlap_wh=(50, 50), overlap_filter="non_max_suppression", iou_threshold=0.1, match_metric="IOS")
detections = slicer(image)

box_annotator = sv.BoxAnnotator()
mask_annotator = sv.MaskAnnotator()
label_annotator = sv.LabelAnnotator()
labels = [f"{class_name} {confidence:.2f}" for class_name, confidence in zip(detections["class_name"], detections.confidence)]

annotated_image = box_annotator.annotate(scene=image, detections=detections)
annotated_image = mask_annotator.annotate(scene=annotated_image, detections=detections)
annotated_image = label_annotator.annotate(scene=annotated_image, detections=detections, labels=labels)
Image.fromarray(annotated_image[..., ::-1]).save("test.jpg")

Any specific deployment considerations

For example, documentation changes, usability, usage/costs, secrets, etc.

Docs

  • Docs updated? What were the changes:

- Added the `match_metric` parameter to the `with_nms` and `with_nmm` methods in the `Detections` class.
- Modified the `box_non_max_suppression` and `box_non_max_merge` functions to support IOS calculation.
- Introduced the `mask_non_max_merge` function for handling non-maximum merging of masks.
- Updated the `box_iou_batch` and `mask_iou_batch` functions to support both IOU and IOS calculations.
- Enhanced the `InferenceSlicer` class to accommodate the IOS matching metric.

### Notes
- IOS (Intersection over Smaller): A variation of IOU that compares intersection over the smaller area, useful in scenarios with varying object sizes.
- Refinements: Improved terminology for clarity and consistency, ensuring technical accuracy.
- Added the `match_metric` parameter to the `with_nms` and `with_nmm` methods in the `Detections` class.
- Modified the `box_non_max_suppression` and `box_non_max_merge` functions to support IOS calculation.
- Introduced the `mask_non_max_merge` function for handling non-maximum merging of masks.
- Updated the `box_iou_batch` and `mask_iou_batch` functions to support both IOU and IOS calculations.
- Enhanced the `InferenceSlicer` class to accommodate the IOS matching metric.

Notes
- IOS (Intersection over Smaller): A variation of IOU that compares intersection over the smaller area, useful in scenarios with varying object sizes.
- Refinements: Improved terminology for clarity and consistency, ensuring technical accuracy.
@CLAassistant
Copy link

CLAassistant commented Jan 9, 2025

CLA assistant check
All committers have signed the CLA.

@SkalskiP
Copy link
Collaborator

SkalskiP commented Jan 9, 2025

Hi @SunHao-AI 👋🏻 thank you so much for your interest in supervision. This looks like a useful feature. I will try to get back to you with a PR review as soon as possible.

@SkalskiP SkalskiP added the priority:high Tasks rated as high priority by Roboflow team label Jan 9, 2025
SunHao-AI and others added 5 commits January 16, 2025 11:09
暂时注释掉创建 GitHub App token 的步骤,可能是因为该步骤不再需要或出现了问题。这个修改不影响其他功能,只是减少了工作流中的一个步骤。
取消注释了 publish-docs.yml 文件中的 Create GitHub App token (mkdocs) 步骤。这个步骤现在将作为工作流程的一部分正常运行,不再被跳过。
Copy link
Contributor

@soumik12345 soumik12345 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @SunHao-AI, thanks for the PR!
I found a bug in your implementation, can you please apply the fix and add some unit tests?

SunHao-AI and others added 4 commits July 7, 2025 15:21
- 移除 merge_inner_detections_objects 函数中的 threshold 和 match_metric 参数
- 新增 merge_inner_detections_objects_without_iou 函数,用于合并检测对象
- 优化非最大值抑制(NMM)的逻辑,提高效率
- 调整 mask_iou_batch 函数的实现,简化代码
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority:high Tasks rated as high priority by Roboflow team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants