BIC and Dynamic Analysis

This repository implements a technique to reduce the search space of bug inducing commits (BICs) accompanying the paper:

Reducing the Search Space of Bug Inducing Commits Using Failure Coverage (FSE'21 IVR Track)

@inproceedings{an2021reducing,
  title={Reducing the search space of bug inducing commits using failure coverage},
  author={An, Gabin and Yoo, Shin},
  booktitle={Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
  pages={1459--1462},
  year={2021}
}

Quick Navigation

• Section 3: Detailed reason for subject exclusion
• Section 3.2.1: Logs produced during the BIC search space reduction process
• Section 3.2.2: Manual Inspection results
• Section 3.2.3: SZZUnleashed results
  • The files ./resources/workspace/issue_list/*.error contain the reason why it is unable to collect the SZZ input data for some subjects.
• All-in-one results (One can reproduce the figures in the paper using the analysis script.)

A. A Guide to Replication

A-1. Prerequisites

• Docker

A-2. (On the host machine) Creating a Docker image & starting a container

docker build -t bic . # this will take a while
docker run -dt --name bic -v $(pwd)/resources/workspace:/root/workspace bic:latest
docker exec -it bic bash

Then, the directory $(pwd)/resources/workspace on the host machine will be mounted into a container (/root/workspace/).

A-3. (On the container) Application on Defects4J

The result files are already provided. The following instructions just guide you through how to replicate it.

A-3a. Initializing the environment

cd ~/workspace
sh initialize.sh

A-3b. Reducing the BIC search space using failure coverage

cd ~/workspace
sh search_BIC.sh [pid] [start_vid] [end_vid]
# sh search_BIC.sh Lang 1 65

See the log for each fault in ~/workspace/logs/search_BIC/{pid}-{vid}b.log. If there is some error while performing the reduction, the error message will be saved to ~/workspace/setup.error.

A-3c. Running SZZUnleashed using collected issue lists

• We have collected issue lists for Defects4J faults, that are the input for SZZUnleashed, using ~/workspace/issue_list/create_issue_list.py.

• The issue information for each fault, {pid}-{vid}b, is saved in the file ~/workspace/issue_list/{pid}/{pid}-{vid}b.json.

  For example, the issue data for Time-10b is as follows:

  {
      "Time-10b": {
          "creationdate": "2013-05-07 14:19:36 +0000",
          "resolutiondate": "2013-06-16 10:28:45 +0000",
          "hash": "3a413d7844c22dc6ddd50bf5d0d55ff3589e47ac",
          "commitdate": "2013-06-16 11:28:26 +0100"
      }
  }

  The hash corresponds to the fixed version commit provided by Defects4J. Find revision.id.fixed in their README to see more details.

• The file ~/workspace/issue_list/{pid}.error contains the detailed reasons for the bugs whose data could not be properly collected.

• Using the issue list file, you can run SZZUnleashed with the following command:

  python3.6 run_szz.sh issue_list/{pid_1} ... issue_list/{pid_n}
  # python3.6 run_szz.sh issue_list/Lang issue_list/Time

  The szz output will be collected in ~/workspace/szz_results.

A-3d. Generating a file containing all experiment results in the log directory

python3.6 log_analyzer.py -d ./logs/search_BIC/ -o ~/workspace/bic_expr.pkl

~/workspace/bic_expr.pkl will contain the overall experimental results in the form of a Pandas Dataframe.

B. Data analysis

B-1. Prerequisites

• Python 3.9.1
• Installing dependencies:

  python -m pip install -r requirements.txt

B-2. Plot drawing

On a host machine, use ./bic_expr_analysis.ipynb to analyze the experiment result file (.pkl) and draw plots.

Make sure that the result file (./resources/workspace/bic_expr.pkl) exists on the host machine. Since we already provide the result file, you can just use the file without replicating the experiment (described in A).

B-3. Manual Inspection (Section 3.2.3 in the paper)

Goto Manual Inspection Results;