Training votenet on synthetic data

[IliasMAOUDJ]

Hello, thank you for this amazing model!

I was wondering if it's possible to train Votenet on a dataset composed of synthetic data only ? I haven't built the dataset yet and I'm still not sure whether I should build an annotated dataset by myself or if synthetic data works fine. If anyone has ever done it, what was the outcome ? Is there anything to pay attention to when building the dataset ?

[orlitany]

We’ve trained multiple successful models on synthetic data using shapenet meshes sampled to point clouds. Note the classes need to be modified accordingly. Also note sampling density — if you use for example 1000 points per model that should work fine. If you sample 10000 points from the entire scene than a scene with just one object would look very different (density-wise) from a scene with 20 objects. Just something to keep be aware of.

On Fri, Mar 12, 2021 at 3:02 AM Ilias MAOUDJ ***@***.***> wrote: Hello, thank you for this amazing model! I was wondering if it's possible to train Votenet on a dataset composed of synthetic data only ? I haven't built the dataset yet and I'm still not sure whether I should build an annotated dataset by myself or if synthetic data works fine. If anyone has ever done it, what was the outcome ? Is there anything to pay attention to when building the dataset ? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#114>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABFYOZUIWQQ6QV6AW264VATTDHYFJANCNFSM4ZCCEVDA> .

-- Or Litany, PhD https://orlitany.github.io/

[IliasMAOUDJ]

Thank you for your answer !

Also, to avoid learning from scratch, I was thinking of trying transfer learning. Is this possible ? Should I try to build my dataset according to the SUNRGBD for example (number of points per scene, number of objects per scene, ...) and then use the pretrained votenet on SUNRGBD? I'm new to 3D computer vision and I haven't found articles dealing with transfer learning.

Edit: I visited your page and watched your new talk on 3D scene understanding from pointclouds (really interesting !)... It seems that transfer learning isn't a good idea/ doesn't provide significant improvement in this context.

[orlitany]

TL may involve additional considerations— if you’re new to the field I would highly recommend starting with a single domain. Training voteNet on simulated data should be fast and accurate

On Fri, Mar 12, 2021 at 7:08 AM Ilias MAOUDJ ***@***.***> wrote: Thank you for your answer ! Also, to avoid learning from scratch, I was thinking of trying transfer learning. Is this possible ? Should I try to build my dataset according to the SUNRGBD for example (number of points per scene, number of objects per scene, ...) and then use the pretrained votenet on SUNRGBD? I'm new to 3D computer vision and I haven't found articles dealing with transfer learning. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#114 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABFYOZXWCLQCF7P6YLIR6NLTDIU5BANCNFSM4ZCCEVDA> .

-- Or Litany, PhD https://orlitany.github.io/

[IliasMAOUDJ]

I see, I will do as you say. Anyway, this is a really exciting topic !

[IliasMAOUDJ]

Hello again @orlitany,

I'm re-opening the issue because I have little knowledge about what size should be my dataset to be trained on VoteNet. I have 3 classes (a tool and 2 bones), might be only the bones later on. I have CAD models for those 3, but I'm not sure how big should be the dataset to be trained from scratch. Do you have an idea of how many scenes I should create ?

Thanks !

[orlitany]

Very hard to say without seeing more details like the variability in geometry and arrangements..I think this matter is specific to your problem so why don’t we close the issue and we can schedule a quick chat to see what makes the most sense for your setting?

On Thu, Mar 18, 2021 at 6:09 AM Ilias MAOUDJ ***@***.***> wrote: Reopened #114 <#114>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#114 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABFYOZRPAYOM56LFMFMAGGDTEH3PFANCNFSM4ZCCEVDA> .

-- Or Litany, PhD https://orlitany.github.io/

[IliasMAOUDJ]

That would be really awesome, yes ! How should I contact you ?