Disable Labelling Half Visible Objects

[anchal-28]

Hi , Is there any way in which we can disable the labelling of the objects that are half visible. Like, for every iteration unity perception will check whether a particular object is fully visible or not and if partially visible, then it wont get labelled in that image frame.
I have tried this using couple of ways but none of them seems to work fine.

[Robotatron]

I'd like to know too. This problem is even worse, because Perception labels an objects that might only be visible for 0.1%, being just couple of pixels. And the 2d bounding box will cover only these 10 pixels, not the whole occluded object

[aryan-mann]

Hey @anchal-28 and @Robotatron, in the latest Perception release although there are ways to approximate how visible/occluded a labeled object is (such as adding keypoints to all objects and seeing how many keypoints are visible out of the total), they are a bit inconvenient to implement.

🎉 However, we have an occlusion labeler in the works that can you give you visibility information. 🎉 You can get the percentage of an object visible in the frame or behind another object etc. that can be used to filter out bounding boxes of partially visible labeled objects!

If this seems of interest, feel free to drop an email to @shounakmitra (shounak.mitra@unity3d.com) and he can take it from there!

[FedericoVasile1]

Thank you for the update aryan, I'm also interested in this occlusion labeler 😄

[aryan-mann]

@FedericoVasile1 Good to know 🎉 @shounakmitra would be super interested in learning more about your use-case if you're open to reaching out!

[methier-CPR]

Hi @aryan-mann, thanks for the good news! I sent an email about a month ago to @shounakmitra asking for timeline estimates for the visibility calculations and still have not gotten an answer. Would you be able to provide an estimate on when we can expect this feature to be released?

[shounakmitra]

Hi Methier, I'll reply to your original email thread. Shounak

…

On Mon, Jun 6, 2022 at 8:16 AM methier-CPR ***@***.***> wrote: Hi @aryan-mann <https://github.com/aryan-mann>, thanks for the good news! I sent an email about a month ago to @shounakmitra <https://github.com/shounakmitra> asking for timeline estimates for the visibility calculations and still have not gotten an answer. Would you be able to provide an estimate on when we can expect this feature to be released? — Reply to this email directly, view it on GitHub <#497 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AYOLPYOB3RKY3D255ELEIS3VNYI5XANCNFSM5XKVYZUA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AnchalT28]

Hi @shounakmitra ,
I had also sent a mail to you regarding the same. But haven't received any reply yet.

[shounakmitra]

Pls check your email Anchal. I’ve requested for a time to chat this week.

…

On Mon, Jun 6, 2022 at 9:22 PM AnchalT28 ***@***.***> wrote: Hi @shounakmitra <https://github.com/shounakmitra> , I had also sent a mail to you regarding the same. But haven't received any reply yet. — Reply to this email directly, view it on GitHub <#497 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AYOLPYL3EGLXASJOIJN7U53VN3FA5ANCNFSM5XKVYZUA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AnchalT28]

Sorry for the misunderstanding

[Robotatron]

Hey @anchal-28 and @Robotatron, in the latest Perception release although there are ways to approximate how visible/occluded a labeled object is (such as adding keypoints to all objects and seeing how many keypoints are visible out of the total), they are a bit inconvenient to implement.

🎉 However, we have an occlusion labeler in the works that can you give you visibility information. 🎉 You can get the percentage of an object visible in the frame or behind another object etc. that can be used to filter out bounding boxes of partially visible labeled objects!

If this seems of interest, feel free to drop an email to @shounakmitra (shounak.mitra@unity3d.com) and he can take it from there!

Sounds great! Will the occlusion labeller also provide a bounding box (and a mask for instance segmentation) when the object is partly occluded? Or is it beyond the scope of the occlusion labeller?

[FedericoVasile1]

@shounakmitra Hi 😄 I sent you an email too

[RuiyuZ]

@Robotatron It is beyond the scope of the occlusion labeler, but the bounding box labeler and the instance segmentation labeler will work as usual which applies to the case when the object is partly occluded