-
Notifications
You must be signed in to change notification settings - Fork 53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Modality fusion implementation question #53
Comments
i have the same question thanks in advance |
Have you guys successively reproduced the model on Nuscenes, I did several experiments but the performance is far away from the report results. And, I also tested the provided weight on Nuscences getting results similar to the reported results. I'd like to know if I forgot to set some arguments. |
This issue is code implementation, not performance. I think open code is not implemented as described in the paper. |
Missing in the code. modality fusion
I think that [fuse_feat = F.relu(feat_cat*feat_wieght) + img_feat] implements the formula in the paper as a code.
|
Thanks for your reply, I will further check this issue. |
I just trained the modified version as you said, and the performance did improve a little bit around 2 on mIoU. I believe there may be some other wrong implements or missing in the released code. And thank you again. |
@kjwkch @LiXiang0021
Currently, the implementation takes the point feature as input directly for Lines 89 to 93 in 80b8646
@yanx27, I would appreciate any suggestions you may have regarding this matter. |
@jaywu109 does the new script changes work for you ? |
hello. I am studying 2DPASS with code.
It seems that the modality fusion implementation is in network/arch_2dpass.py from line 100 to line 105.
In the thesis, bitwise add is specified, but I do not see it in the code, so I ask a question.
Below is the code.
modality fusion
feat_learner = F.relu(self.leanersidx)
feat_cat = torch.cat([img_feat, feat_learner], 1)
feat_cat = self.fcs1idx
feat_weight = torch.sigmoid(self.fcs2idx)
fuse_feat = F.relu(feat_cat * feat_weight)
I think that [fuse_feat = F.relu(feat_cat*feat_wieght) + img_feat] implements the formula in the paper as a code.
Isn't it?
The text was updated successfully, but these errors were encountered: