Advanced BoxAnnotator🤩

[hardikdava]

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Description

Current sv.BoxAnnotator is drawing boxes based on opencv rectangle. It can be improved for this kind of boxes. This looks pretty cool!

• Advanced corner based bounding boxes
• Blur capacity (Direct object blur application)
• Boxes as mask

image source: depthai blogs

Use case

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[SkalskiP]

@hardikdava Haha I see you want to do annotators :) How about we do this first? #49

[hardikdava]

Preliminary result:

[SkalskiP]

Uuuu! Looks cool! 🔥

I drove for a few hours yesterday and thought a lot about our annotators. And I think we can use that opportunity to redesign that feature and make it even more flexible. I think we should split annotators and make them composable. What do you think @hardikdava?

Here is my idea:

import numpy as np

from abc import ABC, abstractmethod

from typing import List


# base annotator

class BaseAnnotator(ABC):

    def annotate(self, scene: np.ndarray, detections: Detections) -> np.ndarray:
        pass


class ComposableAnnotator(ABC):

    def __init__(self, annotators: List[BaseAnnotator]):
        self.annotators = annotators

    def annotate(self, scene: np.ndarray, detections: Detections) -> np.ndarray:
        annotated_image = scene
        for annotator in self.annotators:
            annotated_image = annotator.annotate(scene=scene, detections=detections)
        return annotated_image

# single responsibility annotator

class BoxAnnotator(BaseAnnotator):

    def annotate(self, scene: np.ndarray, detections: Detections) -> np.ndarray:
        pass


class PolygonMaskAnnotator(BaseAnnotator):

    def annotate(self, scene: np.ndarray, detections: Detections) -> np.ndarray:
        pass


class LabelAnnotator(BaseAnnotator):

    def annotate(self, scene: np.ndarray, detections: Detections) -> np.ndarray:
        pass


class PillowLabelAnnotator(BaseAnnotator):

    def annotate(self, scene: np.ndarray, detections: Detections) -> np.ndarray:
        pass


class TrackAnnotator(BaseAnnotator):

    def annotate(self, scene: np.ndarray, detections: Detections) -> np.ndarray:
        pass


class BoxMaskAnnotator(BaseAnnotator):

    def annotate(self, scene: np.ndarray, detections: Detections) -> np.ndarray:
        pass

# high level annotators

class DetectionAnnotator(ComposableAnnotator):

    @classmethod
    def init(cls):
        annotators = [
            BoxAnnotator(),
            LabelAnnotator()
        ]
        return cls(annotators=annotators)


class SegmentationAnnotator(ComposableAnnotator):

    @classmethod
    def init(cls):
        annotators = [
            BoxAnnotator(),
            LabelAnnotator()
        ]
        return cls(annotators=annotators)


class TrackedDetectionAnnotator(ComposableAnnotator):

    @classmethod
    def init(cls):
        annotators = [
            BoxAnnotator(),
            LabelAnnotator(),
            TrackAnnotator()
        ]
        return cls(annotators=annotators)

[hardikdava]

Hey @SkalskiP 👋 , this is actually a nice idea of having computer vision task based annotators along with single responsible annotators. pillow annotators are actually a good idea but I found that pillow is much slower than opencv in terms of drawing operations from my experience. But having it is a good option. Since there are a lot possibilites of drawing for example it allows to use custom fonts which opencv doesn't allow. It will be a feature for future release. I would suggest to create an issue for this for now. We should first try to solve the existing issues.

[SkalskiP]

Hi @hardikdava 👋🏻 ! I know pillow is slower than opencv, but opencv only allows you to use ascii characters in the text. We already had a lot of voices from the community that they would like to add text in Arabic or Chinese.

But I mostly mean to give people base annotators that they could freely combine.

[hardikdava]

Hi @SkalskiP 👋 , I am 💯 agree with you about adding pillow. It is good to have.

[SkalskiP]

@hardikdava how about the overall idea? 💡

[hardikdava]

I would keep this as simple as possible. User can choose between cv and pillow with one flag. Creating different Annotators will create lot of confusion in usage and practice.

class BoxAnnotator(BaseAnnotator):

    def annotate(self, scene: np.ndarray, detections: Detectionsm, use_pillow: bool=False) -> np.ndarray:
        if use_pillow:
            scene = self._annotate_pillow()
            return scene
        else:
            scene = self._annotate_cv()
            return scene
    def _annotate_cv(self):
        pass
     def _annotate_pillow(self):
        pass

[SkalskiP]

I don’t want to use pillow for drawing boxes. I want to use pillow only for text annotation and I want to separate drawing labels from drawing boxes.

[hardikdava]

@SkalskiP Overall, this is really good idea and your suggested APIs are good for future. It is highly scalabel.

[SkalskiP]

Exactly. Problem is that we would need to do it in the way that people could migrate from old API to new API.

[hardikdava]

I do not think of any problems of migration with the suggested API. This will work without any issue.

[SkalskiP]

@hardikdava I'll create the initial PR to set up the structure, and we will migrate all annotators one by one. Okey?

[hardikdava]

@SkalskiP okay. I will be waiting for the issue.

[SkalskiP]

@hardikdava can you open PR with this annotator into the feature/redesigned_annotators branch?

[hardikdava]

@hardikdava can you open PR with this annotator into the feature/redesigned_annotators branch?

Of course. I will make a PR.

[SkalskiP]

Hi @hardikdava 👋🏻! Sorry I move so slowly. I'm doing a lot of stuff other than supervision this week. :/

[hardikdava]

@SkalskiP understood. It's fine. I am working on Annotators and trying to answer in the issues. Take your time and later try to test and merge the functions.

[maddust]

Just an idea I added to my project to have a visual bbox queue of when an object triggers a PolygonZone or LineZone , also check the crop slideshow on the bottom

[SkalskiP]

@maddust 🔥! This looks amazing! You are truly a supervision power user!

[SkalskiP]

This task is finally completed with: https://github.com/roboflow/supervision/releases/tag/0.15.0 🔥