Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

BugsJS bug dataset

This repository contains the data files and the utility framework of the BugsJS benchmark.


This benchmark includes bugs from the following 10 projects:

  • Bower
  • Eslint
  • Express
  • Hessian.js
  • Hexo
  • Karma
  • Mongoose
  • Node-redis
  • Pencilblue
  • Shields


The tests of the Node-redis project require ipv6 support.

The per-test coverage data of the ESlint project is located in a separate storage due to its size (3GB).

ESlint coverage data

Files in the repository

In the Projects directory, there is a subdirectory for each project which contains the following files:

  • [project]_bugs.csv: pre-calculated information about the tests of each bug
  • [project]_commands.csv: the necessary test runner commands for each bug
  • [project]_issues.bin: the bug reports of the subject project save in the Google's Protocol Buffers format (for the structure see the project.proto file)


The framework's command-line interface includes the following commands:

  • info: Prints out information about a given bug.
  • checkout: Checks-out the source code for a given bug.
  • test: Runs all tests for a given bug and measures the test coverage.
  • per-test: Runs each test individually and measures the per-test coverage for a given bug.

For the checkout, test, and per-test commands, the user can specify the desired code revision:

  • buggy: The parent commit of the revision in which the bug was fixed.
  • fixed-only-test-change: A revision containing only the tests introduced in the bug fixing commit, applied to the buggy revision.
  • fixed: A revision containing both the cleaned fix and the newly added tests.

Example command:

python3 -p Bower -b 1 -t checkout -v fixed -o output/