Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Number of classes of relations in VG #29

Open
bowen-upenn opened this issue Jul 16, 2023 · 0 comments
Open

Number of classes of relations in VG #29

bowen-upenn opened this issue Jul 16, 2023 · 0 comments

Comments

@bowen-upenn
Copy link

bowen-upenn commented Jul 16, 2023

May I kindly ask why each sample in outputs_class_rel = self.rel_class_embed(torch.cat((hs_sub, hs_obj, so_masks), dim=-1)) at models/reltr.py:111 has dim 52, instead of 51? The processed VG has 50 relation classes, so I assume the dim should be 51 with an additional no relation ('background') class.

Besides, it can be seen from models/reltr.py:213-214
Count the number of predictions that are NOT "no-object" (which is the last class)
card_pred = (pred_logits.argmax(-1) != pred_logits.shape[-1] - 1).sum(1)
that the last class represents the "no-object" class.

However, in the colab notebook, you said
REL_CLASSES = ['background', 'above', 'across', 'against', 'along', 'and', 'at', 'attached to', 'behind',
'belonging to', 'between', 'carrying', 'covered in', 'covering', 'eating', 'flying in', 'for',
'from', 'growing on', 'hanging from', 'has', 'holding', 'in', 'in front of', 'laying on',
'looking at', 'lying on', 'made of', 'mounted on', 'near', 'of', 'on', 'on back of', 'over',
'painted on', 'parked on', 'part of', 'playing', 'riding', 'says', 'sitting on', 'standing on',
'to', 'under', 'using', 'walking in', 'walking on', 'watching', 'wearing', 'wears', 'with']

I notice that REL_CLASSES has a length of 51, not 52, and 'background' is at the index 0, not at the last index.
Is this REL_CLASSES in colab the label indices you use in your training code (in data/vg/rel.json)? Because I am working on re-organizing the dataset labels for my own project, I need to know the exact ordering of these label indices. Thanks for your assistance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant