No description, website, or topics provided.
Python Java PHP HTML Shell CSS Other
Clone or download
Latest commit 98f7ee5 Jul 11, 2018

MUBench Logo


MUBench (pronounced "Moo Bench") is an automated benchmark for API-misuse detectors, based on the MUBench benchmarking dataset. If you encounter any problems using MUBench, please report them to us. If you have any questions, please contact Sven Amann.

CI Status: CI Status



We provide instructions to reproduce the MUBench experiments presented in the above publications.

Getting Started

With MUBench, you may run different API-misuse detectors in a number of experiments to determine their precision and recall. To this end, MUBench provides a curated dataset of real-world projects and known misuses. In each experiment run, the respective detector emits findings which you need to review manually. To this end, MUBench publishes (a subset of) the findings to a review website. After you completed your reviews, the site automatically computes experiment statistics.


  1. Setup a review (web)site to publish detector findings to
  2. Setup the experiment pipeline to run experiments


  1. Run experiments, using ./mubench run
  2. Publish detector findings to your review site, using ./mubench publish


We want MUBench to grow, so please be welcome to


All software provided in this repository is subject to the CRAPL license.

The detectors included in MuBench are subject to the licensing of their respective creators. See the information in the detectors' folders.

The projects referenced in the MuBench dataset are subject to their respective licenses.

The project artwork is subject to the Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0).