COM label3D

[yuan0821]

Hi. After formating the data file, we get structured camera parameters and sync files. From my understanding, if we want to use your COM and DANNCE networks, we should first get our hand labels for our own mice.
Now I want to ask how to do the next steps for hand labels. Many thanks if you could give some suggestions.

1. Should I generate the hand-labelled COM data for COM network training using dannce\Label3D\skeletons\com.mat file? If so, is there any description of how to label the COM?
2. Or do I need to create our label3d_dannce.mat, which used the file of dannce\Label3D\skeletons\rat23.mat to create hand-label landmarks for DANNCE network?

[data-hound]

Hi @yuan0821

For training on custom data, you need both COM and DANNCE (keypoint) labels. So,

1. As a first step, you will have to label the COM for a few frames and train the COM network on the labelled frames. Then, generate COM predictions - these COM predictions will be used in DANNCE training
2. In the second step, you will have to label the desired set of keypoints, and train DANNCE network - this will create the label3d_dannce.mat file.

For the first step, you need to use the \Label3D\skeletons\com.mat as the skeleton file. Check Label3D Readme for details on how to use it to label COM and keypoints.

For the second step, you need to use Label3D again to make a few labelled frames, and then train on those. If you are using rats, you can use the rat23.mat skeleton. If you are using mice, you can use the following file: https://github.com/spoonsso/dannce/blob/master/configs/mouse22_skeleton.mat

[yuan0821]

Hi, thanks for your reply. I still wonder where can I find the pre-trained 3-camera COM weight network before finetune. Thank you. Comments #62 has given the pre-trained 3-dannce weight, is it the same weight used in com-train and com-predict? Thank you!

[spoonsso]

@yuan0821 the COM network works for any number of views -- frames are processed individually.