Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

请问你们用的是什么算力呀？ #13

Closed
Tonyboy999 opened this issue Nov 20, 2021 · 7 comments
Closed

请问你们用的是什么算力呀？ #13

Tonyboy999 opened this issue Nov 20, 2021 · 7 comments

Comments

@Tonyboy999
Copy link

原文用了128核的TPUv3，完全是我企及不到的算力。请问你们用了什么算力啊，我评估一下我手上的算力配不配做预训练
@pengzhiliang
Copy link
Owner

3090/V100

@Tonyboy999
Copy link
Author

多少张呀

@launchauto
Copy link

多少张呀

8卡3090或者V100，单卡batchsiz512, 保证总batchsize是4096就和paper描述的mae-vit-base配置一致。
我这边用了64卡 A100-40G，单卡batchsize 32, 总batchsize4096 mae-beit-base在ImageNet1k上训练了1600个epoch。

@pengzhiliang
Copy link
Owner

有钱人，我只用了8张v100，可以保证batchsize 4096

@leeyegy
Copy link

leeyegy commented Nov 23, 2021

请问八卡v100需要跑多久呀

@leeyegy
Copy link

leeyegy commented Nov 23, 2021

多少张呀

8卡3090或者V100，单卡batchsiz512, 保证总batchsize是4096就和paper描述的mae-vit-base配置一致。 我这边用了64卡 A100-40G，单卡batchsize 32, 总batchsize4096 mae-beit-base在ImageNet1k上训练了1600个epoch。

话说32*64 卡不是batch size只有2048嘛？

@slchenchn
Copy link

这小batchsize影响大吗？穷人没那么多卡呜呜

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@launchauto @slchenchn @pengzhiliang @leeyegy @Tonyboy999