Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kind request for updating MT-Bench leaderboards with Qwen1.5-Chat series #3009

Closed
Lukeming-tsinghua opened this issue Feb 4, 2024 · 6 comments

Comments

@Lukeming-tsinghua
Copy link

Lukeming-tsinghua commented Feb 4, 2024

Hi LM-Sys team, we would like to present the generation results and self-report scores of Qwen1.5-7B-Chat, Qwen1.5-14B-Chat, and Qwen1.5-72B-Chat on MT-Bench. Could you kindly help us verify them and update the leaderboard to add the Qwen2-beta-Chat series? The generation results are attached in the zip file.

qwen2-beta-mtbench-results.zip

Model Self-report Score
Qwen1.5-72B-Chat 8.67
Qwen1.5-14B-Chat 7.99
Qwen1.5-7B-Chat 7.58
@Lukeming-tsinghua Lukeming-tsinghua changed the title Kind request for updating MT-Bench leaderboards with Qwen2-Beta-Chat series Kind request for updating MT-Bench leaderboards with Qwen2-beta-Chat series Feb 4, 2024
@infwinston
Copy link
Member

thanks! is qwen2-beta publicly available now?

@Lukeming-tsinghua Lukeming-tsinghua changed the title Kind request for updating MT-Bench leaderboards with Qwen2-beta-Chat series Kind request for updating MT-Bench leaderboards with Qwen1.5-Chat series Feb 5, 2024
@Lukeming-tsinghua
Copy link
Author

Thanks for your help!

We plan to make the HF repos public and begin to provide APIs today. By the way, we use the official name Qwen1.5-Chat instead of the previous Qwen2-beta in the zip file. Sorry for the confusion we made.

@OpenJarvisAI
Copy link

@Lukeming-tsinghua Does there only same 7b 14b 70b in Qwen1.5?

@gblazex
Copy link

gblazex commented Feb 5, 2024

great work guys!
will this be up on Human Arena as well? 🤞👀

@infwinston
Copy link
Member

yes we're testing it out. will soon launch.

@gblazex
Copy link

gblazex commented Feb 16, 2024

this can be closed i think

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants