You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi,
I'm trying to train DINOv2 with the ViT-S backbone. I've noticed that the model training crashes if drop_path_rate > 0. In particular, I get the following error
RuntimeError: Expected source.size(1) % kNumElementsPerBlock == 0 to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.)
As recommended in issue #108, one need to set drop_path_rate = 0 when using the ViT-S backbone. What is the reason for this behaviour? In the original DINO, the drop_path_rate = 0.1 for both ViT-S and ViT-B.
The text was updated successfully, but these errors were encountered:
Yes, that worked for me, thank you! Except I had to add back the fmha import since it seems to have been mistakenly removed from the updated import statement.
Hi I have the same issue with drop_path_rate>0 for VIT-SMALL, can you let me know what is the reason and which commit you fix this issue? I am using the latest version of repo so far and issue still exist apperantly
Hi,
I'm trying to train DINOv2 with the ViT-S backbone. I've noticed that the model training crashes if
drop_path_rate
> 0. In particular, I get the following errorwith the traceback going all the way to line 173
As recommended in issue #108, one need to set
drop_path_rate = 0
when using the ViT-S backbone. What is the reason for this behaviour? In the original DINO, thedrop_path_rate = 0.1
for both ViT-S and ViT-B.The text was updated successfully, but these errors were encountered: