Question about Experiments on NOCS-REAL 275 dataset

[taeyeopl]

Thanks for sharing good work.
I have some questions related to your work.

Q1. Have you tried experiments on NOCS-REAL 275 dataset [1] using propose the T-NOCS??
The representation of NOCS comes from the [1] paper and the dataset consists of video sequences dataset.
If I were you, I would have considered experiments with an existing real world video dataset (ex, NOCS-REAL 275).
If you did, I'm curious what difficulties you encountered. If not, I wonder why you didn't consider the experiment.

Q2. Can you explain the difference between GT-NOCS and GT-TNOCS??
May I understand that GT-TNOCS generated by the union of K partial of GT-NOCS??

Q3. In the limitations and future work part, It mentioned that "dense supervision of T-NOCS labels is may not available for real data".
As I understood, the NOCS-REAL training dataset [1] already has T-NOCS labels.
Can you explain clearly why you mentioned that may not be available for real data??
Have I missed something??

[1] Wang, He, et al. "Normalized object coordinate space for category-level 6d object pose and size estimation." CVPR. 2019.

• NOCS-REAL 275: evaluation set
• NOCS-REAl training dataset: training set

[davrempe]

Hello, thanks for your interest.

Q1/Q3) We did not perform experiments on any real-world data due to the lack of readily-available data. I have not used the datasets in [1], however I think NOCS-REAL is rather small (compared to the synthetic data we used in CaSPR) and intended for fine-tuning to improve the sim to real gap. This means we would likely also need some synthetic data to fully train CaSPR; and I'm not certain but it looks like the CAMERA dataset [1] is just images, not videos. Besides NOCS-REAL, I am not aware of other real-world datasets with dense NOCS labels, which is why we discussed this as a limitation.

Q2) GT-NOCS are the 3D canonical points for a single frame, whereas ground truth T-NOCS (that we use for training) is a sequence of NOCS where each point also has a timestamp, so it is 4D. But if you were to remove the timestamp, then yes you can see GT-TNOCS as the union of a sequence of GT-NOCS.