Android Bench is a benchmark for evaluating Large Language Models (LLMs) on tasks specific for Android development.
We are continuoulsy working on making the Android Bench better. That's why we are sharing an opportunity with you to contribute your own model evaluations! By contributing your local runs, you help maintain a healthy ecosystem intake pipeline and establish Android Bench as the referent for generative quality Android code.
Interested in proposing or implementing tasks for Android Bench? Check out the community-dataset.
Important
For now, submissions will be accepted with no guarantee of publication.
We are currently looking for new model evaluations to enhance our leaderboard!
Interested in the process? Our detailed guide on how to contribute, run evaluations, and format your data is located under CONTRIBUTING.md.
Learn more information about Android Bench on the Android Bench website.
Learn more about the Harbor framework and what tools it provides.
We are using Apache 2.0 License - see LICENSE for more information.
Contributions to this project must be accompanied by a Contributor License Agreement (CLA). You (or your employer) retain the copyright to your contribution; this simply gives us permission to use and redistribute your contributions as part of the project.
If you or your current employer have already signed the Google CLA (even if it was for a different project), you probably don't need to do it again.
Visit https://cla.developers.google.com/ to see your current agreements or to sign a new one.
This project follows Google's Open Source Community Guidelines.