Paper

Misoo Kim, Eunseok Lee, "Are Datasets for Information Retrieval-based Bug Localization Techniques Trustworthy?", Empirical Software Engineering (ESE), Vol.26, No.3, pp.1-66, Mar.2021

Prepare the Bench4BL

• https://github.com/exatoa/Bench4BL
• J. Lee, et al. Bench4bl: reproducibility study on the performance of ir-based bug localization, ISSTA'18

File Description

./config

• Bench4BL = path of original bench4bl dataset
• output = path to save the new bug repository

datagen.core.bench4bl.DataGen.java

• Create a new bug repository by extracting the buggy methods based on the bug repositories provided by Bench4BL
• Classify the source files into production and test files, and create index files using Lucene

./bench4bl/spring/shdp/

• sample data for testing
• bugrepo: original bug repository from the Bench4BL
• sources: original source file collection from the Bench4BL
• gitrepo: git repository

./output/spring/shdp/

• output data from the DataGen.java
• bug: new bug repository
• sources: new source file collection having already index files

irbl.core.IRBL.java

• based on new bug repository generated by DataGen.java
• Evaluate the performance of the IRBL based on Lucene
• Evaluation options: original, correct ground-truth files (GTF), production bug localization (PBL), test bug localization (TBL)

discussion.classifier.java

• Production and Test Bug Classification based on Naive Bayes
• option: Binary Naive Bayes, Original Naive Bayes

./classification_data/

• Training and Testing Dataset for eclipse.platform and eclipse.jdt
• Class: POS: TB=10 / PB=0, NEG: TB=0 / PB=10

Our Research Results

• https://drive.google.com/open?id=1AKC9sydf2-IiGEeZB94Mx42qsNJaDw1Z

• ./Tables/ : All investigation and experimental results

  • Our researh results is based on below researches. We are most grateful to authors
  • J.Zhou'12: J.Zhou, et al. Where should the bugs be fixed? more accurate information retrieval-based bug localization based on bug reports, ICSE'12
  • B. Dit'13, Feature location in source code: ataxonomy and survey, Journal of software: Evoluation and Process
  • B. Sisman'13, Assisting code search with automatic query reformulation for buglocalization, MSR'13
  • L. Moreno'14, On the use of stack traces to improvetext retrieval-based bug localization, ICSME'14
  • X. Ye'14, Learning to rank relevant files for bug reports using domain knowledge, FSE'14
  • Bench4BL, Bench4bl: reproducibility study on the performance of ir-based bug localization, ISSTA'18

• 1_Bench4BL_bugs.Method.zip: new bug repository extracted by DataGen.java

• 2_Bench4BL_bugs_Type.zip: a set of pairs with bug id and bug type

• 3_Exp_Results_Tools.zip: All detail experimental results for six tools (we used these tools from the Bench4BL)

  • BugLocator: J.Zhou, et al. Where should the bugs be fixed? more accurate information retrieval-based bug localization based on bug reports, ICSE'12
  • BLUiR: R.K.Saha, Improving bug localization using structured information retrieval, ASE'13
  • BRTracer: C.P.Wong, et al. Boosting bug-report-oriented fault localization with segmentation and stack-trace analysis, ICSME'14
  • AmaLgam: S.Wang, Version history, similar report, and structure: Putting them together for improved bug localization, ICPC'14
  • BLIA: K.C. Youm, et al. Bug localization based on code change histories and bug reports, APSEC'15
  • LOCUS: M.Wen, et al. Locus: Locating bugs from software changes, ASE'16

Contributor

• Misoo Kim (misoo12@skku.edu, hanul1546@gmail.com)
• Eunseok Lee (leees@skku.edu)