Process to create sub labels for items with 2D obj detection?

[lessw2020]

Hi - I'm working with the perception package and wanted to ask what is the recommended process to do 2D detection labelling for parts within a given object?
Specifically, the tutorial shows how to detect objects as single units (i.e. this is a whipping cream container, this is a tea container, etc).
For our work we need to label/detect not just the object, but also parts within an object.. - so for the whipping cream example, imagine you need to detect the whipping cream carton, but also the expiry label, the cap on it, the brand label etc.
example with the whipping cream carton:

Can you advise how to set this up? I've made singular fbx assets and not sure how to define labels for not just the object, but for internal parts on the object.

Thanks very much. Quite excited to move more and more of our dataset creation work to synthetic generation!

[JonathanHUnity]

Labeling applies to GameObjects in their entirety, so if you want different labels you need different GameObjects.

There are a couple of approaches to splitting up your object.

The first is to split your object up into multiple objects in your 3d editor and then export the set of objects into the fbx. Here is an example of two cubes in Blender, and those same cubes in Unity:


Since there are two child objects in the top-level "TwoCubes.fbx", you can add a Labeling component to each one.

The other approach is to add invisible geometry to your objects in Unity and then label that.

What I did here is the following:

• Open the whipping cream prefab
• Add a child Cylinder and orient it to cover only the cap
• Create a new material - call it "transparent" - and set "Surface Type" to transparent and change the "Base Map" color's alpha to 0
• Drag the material onto the Cylinder
• Add labeling to the cylinder
  

Repeat this with a Quad to cover the label

Labeling applies to GameObjects. Even transparent geometry is captured in the ground truth, so you can use that to your advantage.

Hope this helps!

[lessw2020]

Thanks very much @JonathanHUnity - that helps tremendously!
Also the gifs for showing the process of invisible labelling are super helpful for explaining the process.
Best regards,
Less

[JonathanHUnity]

Happy to help :)