Some misunderstanding about the heat map using to predict Relationship

[kent252]

Hello, thank you very much for providing the code. However, in the paper, you said that "The predicate probability pˆ𝑝𝑟𝑑 is predicted by a multi-layer perceptron concatenating the corresponding subject representation, object representation, and spatial feature vector, which can be formulated as:
pˆ𝑝𝑟𝑑 = softmax(MLP([Q𝑠,Q𝑜,V𝑠𝑝𝑎]))."
But in the source code, I don't see this concatenating, I just see that you used sub heatmap for prediction. Could you explain it to me. Thank you very much

• In RelTR model:
  
• and in Transformer:
  

[yrcong]

Hello, please check Line 111 in relater.py

outputs_class_rel = self.rel_class_embed(torch.cat((hs_sub, hs_obj, so_masks), dim=-1))

[kent252]

Thanks for your immediate response. I appreciate that very much