Replication Package for "An Empirical Study of Obsolete Answers on Stack Overflow"

Haoxiang Zhang, Shaowei Wang, Tse-Hsun (Peter) Chen, Ying Zou, and Ahmed E. Hassan
IEEE Transactions on Software Engineering

Abstract: Stack Overflow accumulates an enormous amount of software engineering knowledge. However, as time passes, certain knowledge in answers may become obsolete. Such obsolete answers, if not identified or documented clearly, may mislead answer seekers and cause unexpected problems (e.g., using an out-dated security protocol). In this paper, we investigate how the knowledge in answers becomes obsolete and identify the characteristics of such obsolete answers. We find that: 1) More than half of the obsolete answers (58.4%) were probably already obsolete when they were first posted. 2) When an obsolete answer is observed, only a small proportion (20.5%) of such answers are ever updated. 3) Answers to questions in certain tags (e.g., node.js, ajax, android, and objective-c) are more likely to become obsolete. Our findings suggest that Stack Overflow should develop mechanisms to encourage the whole community to maintain answers (to avoid obsolete answers) and answer seekers are encouraged to carefully go through all information (e.g., comments) in answer threads.

BibTeX

@ARTICLE{Zhang2019,
author={H. Zhang and S. Wang and T. Chen and Y. Zou, and A. E. Hassan},
title={An Empirical Study of Obsolete Answers on Stack Overflow},
journal={IEEE Transactions on Software Engineering},
year={2019},
doi={10.5281/zenodo.1320311},
url={https://doi.org/10.5281/zenodo.1320311}
}

Data

• Quantitative Data:
  • all 58201 observations of obsolete answers (QuestionID, AnswerID, CommentID, CommentText)
• Qualitative Data:
  • all 669 obsolete answers in RQs (with labels)