Some questions about loss calculation

[LZDSJTU]

Hello, I read the code on calculating the loss and I have some questions.

1. line 167:
   group_mat_label = tf.matmul(pts_group_label,tf.transpose(pts_group_label, perm=[0, 2, 1]))
   I do not understand the meaning of this operation. Could you give an simple example for it.

2. line 179~184:
   diffgroup_samesem_mat_label = tf.multiply(diffgroup_mat_label, samesem_mat_label)
   diffgroup_diffsem_mat_label = tf.multiply(diffgroup_mat_label, diffsem_mat_label)

   num_samegroup = tf.reduce_sum(samegroup_mat_label)
   num_diffgroup_samesem = tf.reduce_sum(diffgroup_samesem_mat_label)
   num_diffgroup_diffsem = tf.reduce_sum(diffgroup_diffsem_mat_label)
   I am not clear about these code.

3. line 191:
   pos = tf.multiply(samegroup_mat_label, pred_simmat)
   I know that it want to compute the distance in the same group. But I do not know why this sentence can work. Could you provide an simple example or some equation?

4. line 197:
   group_mask_weight = tf.matmul(group_mask, tf.transpose(group_mask, perm=[0, 2, 1]))
   I think that maybe this is the same opeartion as line 167, but I cannot undestand

5. line 209~210:
   Pr_obj = tf.reduce_sum(pts_semseg_label,axis=2)
   Pr_obj = tf.cast(Pr_obj, tf.float32)
   I guess that all element in Pr_obj may be 1. So what the function of it as a weight?

[LZDSJTU]

Thank you very much. And you can choose any of them to answer.

[BenjiDayan]

Hi @LZDSJTU ,
which file are you looking at?

Edit: nvm it's model.py

[BenjiDayan]

@LZDSJTU the only one I've worked out so far since the code includes a comment:

1. line 167:
   pts_group_label is a Batch x num_points x num_groups ndarray, I think one hot style so for each point if there are 50 groups then it has 49 0s and one 1. Transposed it's B x NG x NP, so multiplying with itself you get B x NP x NP.
   As the comment says # BxNxN: (i,j) if i and j in the same group
   So the B x NP x NP has 0s everywhere where point (i, j) not in same group, and 1s where they are.

[densechen]

Hi, @Ziggareto
When I test the pre-trained model in area 1, I even got the 1
pre-trained model test in area 1 AP in 6 cate. What's more, this also happened in area 5.
Do I set the wrong parameters? And Could you please provide the pergroupthres.txt file to us?

Instance Segmentation AP: [0.77272727 0.96409938 0.58364528 0.9 0.27223911 1.
0.77928571 0.67933635 0.70536131 0.48571429 0.59607843 0.8
0.32612594]
Instance Segmentation mAP: 0.6818933128790516
Semantic Segmentation IoU: [0.98613754 0.98480023 0.93936964 0.95665216 0.82676315 0.95118879
0.9408741 0.89464145 0.88190051 0.95819404 0.83323726 0.91617618
0.88444496]
Semantic Segmentation Acc: %f 0.973563616147

Looking forward your reply.

[BenjiDayan]

Hi @LittleLampChen I haven't been following for ages, but I think pergroupthres.txt is generated by valid.py? Have you run that already?

[PatrickFeng]

Issue 1:
@Ziggareto has given the point.

Issue 2:
diffgroup_samesem_mat_label reprensents points of same semantic class but not same instance. And diffgroup_diffsem_mat_label represents points of neither same class nor same instance.

Issue 3:
pred_simmat represents ||Fsim1 - Fsim2||^2 in the paper. It is the calculation of Lsim.

Issue 4:
Referd from the paper:
"Since the semantic segmentation network is actually wrongly trying to bring pairs of points in our second similarity class closer together in feature space, we also add an α > 1 term to increase the weight of our loss to dominate the gradient from the semantic segmentation output branch."

Issue 5:
When a point belongs to the background, the corresponding row of Pr_obj is 0 not 1. You can refer to section 3.1 of the paper