You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, I like your good work!
I am wondering have you tried not to crop the gradient of $f^{\alpha}$ in eq7?
If you crop the gradient, it seems like the pertaining of the point branch cannot learn anything from the image branch.
The text was updated successfully, but these errors were encountered:
Please remember the goal of SimIPU is to learn spatial-aware visual representation. The point branch is only used to capture spatial information and transfer it to the image encoder during the pre-training stage. After that, the point branch is abandoned, and we only use the image encoder to fine-tune on downstream tasks. So during pre-training, there is no need for the point branch to learn anything from the image branch. The knowledge learned from the equivalent of global transformation is the key insight of the point branch.
Indeed, the point branch can learn some semantic, color, or texture information from the image branch, which can benefit the point branch pre-training. While this is not our goal in SimIPU, it is interesting to explore some LIDAR encoder pre-training strategies based on our SimIPU pipeline. Feel free to further discussion.
Hi, I like your good work!$f^{\alpha}$ in eq7?
I am wondering have you tried not to crop the gradient of
If you crop the gradient, it seems like the pertaining of the point branch cannot learn anything from the image branch.
The text was updated successfully, but these errors were encountered: