Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QA] Number of training tokens #97

Closed
RicardoDominguez opened this issue Feb 27, 2024 · 6 comments
Closed

[QA] Number of training tokens #97

RicardoDominguez opened this issue Feb 27, 2024 · 6 comments
Labels

Comments

@RicardoDominguez
Copy link

Hi,

What is the number of training tokens for the base models Qwen 1.5 0.5B, 1.8B, 4B, 7B, 14B, 72B models?

Thanks for the great work!

@JustinLin610
Copy link
Member

2.4, 2.4, 2.4, 4, 4, 3 trillion tokens

@RicardoDominguez
Copy link
Author

Many thanks!

@guijinSON
Copy link

Thanks for the great work. can you also share how many tokens the 110B model was trained on?

@guijinSON
Copy link

and the 32 B model as well.

@RicardoDominguez
Copy link
Author

Yes, it would be great to know the number of tokens of both the 110B and 32B

Copy link

github-actions bot commented Jul 4, 2024

This issue has been automatically marked as inactive due to lack of recent activity. Should you believe it remains unresolved and warrants attention, kindly leave a comment on this thread.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants