Skip to content

Reproducibility Kit

Jump to bottom
Antonela edited this page Oct 29, 2019 · 6 revisions

This repository is a companion page for the paper:

“An Evolutionary Analysis of a Technical Debt Index for Identifying Architecture-sensitive Smells”. (2019)

Abstract

Technical debt indexes are a metric-based approach for assessing the quality of a software system, which focuses on (and tries to quantify) different degradation symptoms. To be of practical usage, a technical debt index is expected to guide engineers in identifying issues that might negatively affect system evolution and maintenance. In this study, we consider an index proposed in the literature, called Architectural Debt Index (ADI), which is based on the detection of architectural smells and the computation of their severity and other features. On this basis, we perform a sensitivity analysis of the ADI values in order to determine the most influential elements (e.g., smells, packages) causing ADI variations over system versions. Also, we propose a decomposition of the ADI into its constitutive elements. By means of these decomposition and sensitivity analysis techniques for a given debt index, we can prioritize key architectural smells that, due to their variations or instability, should be brought to the engineers' attention. An initial evaluation with a range of versions for three open-source systems has shown correlations between the smells detected and ranked with our approach and maintenance/development issues reported for the systems.

Replication material

This repository contains the datasets for the 3 analyzed systems, namely:

Files

For each system, the following artefacts are provided:

  1. Smells and ADI features computed by Arcan for the different system versions

    • ArchitecturalDebtEstimation_smell_evolution_<system -version-span>_ (CSV file): It tracks the behavior of smells across system versions in terms of the four features on which the ADI score is based on (SeverityScore, PageRank, TotNumOfSetOfElements and NumSetOfElement). Each row of the file represents a smell and a feature (thus, a smell is described in four rows), and each column represents a system version.

  2. Results of the Sobol method for key smells

    • parameters file: each variable (smell) and its range of value
    • parameterValues file: the values sampled by the sensitivity method for the parameters
    • objectiveValues file: the global ADI computed out of the parameter values
    • sobolIndices file: the Sobol indices for the parameter and objective values
    • sobolSmellRanking: the ranking of key smells resulting from the Sobol indices

  3. Top-level packages for the different system versions

    • _TXT file: it contains all system packages for all the analyzed versions. Only the last version was used in the experiments.

  4. Results of the Sobol method for key packages

    • parameters file: each variable (smell) and its range of value
    • parameterValues file: the values sampled by the sensitivity method for the parameters
    • objectiveValues file: the global ADI computed out of the parameter values
    • sobolIndices file: the Sobol indices for the parameter and objective values
    • sobolPackageRanking: the ranking of key packages resulting from the Sobol indices

  5. Analysis of issues for the different versions The corresponding files are organized into folders named <project_name>_jira_issues that contains the csv files downloaded from the Jira platform. A single csv file contains the issues of a specific package of the analyzed project. Each file reports for each Jira issue the following information:

    • _Issue Type
    • _Patch available: a custom field indicating whether there is an available patch for the project
    • _Issue Key: issue name
    • _Issue id: issue identifier
    • _Parent id: the id of the eventual parent issue
    • _Summary: a brief description of the issues
    • _Assignee
    • _Reporter
    • _Priority
    • _Status: the current status of the issue
    • _Resolution
    • _Created: date of creation of the issue on the platform
    • _Updated: date of last update of the issue
    • _Due date: eventual issue expiring date

The files named smell_severity_issues*: contains the counts of the Jira issues used to assess whether key smells are related to actual design problems.

  • smell_severity_issues_total: contains the issue count for every type of AS and for every analyzed project. The columns of the file report the identifier of the AS, the rows report namely:
  • Issues: the number of issues
  • median<project_name>: the median value of issues for each analyzed project
  • cover<project_name>: the percentage of key AS related to design problems for each analyzed project.
  • smell_severity_issues_cd<project_name>: contains the issue count for the Cyclic Dependency smell, used to compute the mean of the number of issues for each smell instance. There is one file per analyzed project. Each file contains the issue count for each instance of CD. The columns report the name of the package involved, the rows indicate namely:
  • Issues: the number of issues
  • Blocker, Critical, Major, Minor,Trivial: the number of issues of a specific priority level
  • mean_issues: the mean number of issues of the CD instance
  • median_issues: the median number of issues of the CD instance

The file containing Camel results has more rows, since a detailed analysis was conducted on specific instances of this project, namely:

  • in_other_smell: the number of AS instances in which the package appears other than CD
  • mean_blocker/critical/major/minor/trivial: the mean number of issues of a specific
  • granularity level

The files are organized according to the 3 projects analyzed in the study. A short description of smells found in each version is included.

System Versions used

Apache Camel

  • apache-camel-2.2.0 CD=48 UD=10 HL=3
  • apache-camel-2.3.0 CD=49 UD=10 HL=3
  • apache-camel-2.4.0 CD=49 UD=10 HL=3
  • apache-camel-2.5.0 CD=49 UD=10 HL=3
  • apache-camel-2.6.0 CD=49 UD=10 HL=3
  • apache-camel-2.7.0 CD=26 UD=6 HL=3
  • apache-camel-2.8.0 CD=78 UD=11 HL=3
  • apache-camel-2.9.0 CD=90 UD=11 HL=3
  • apache-camel-2.10.0 CD=92 UD=11 HL=3
  • apache-camel-2.11.0 CD=97 UD=11 HL=3
  • apache-camel-2.12.0 CD=97 UD=11 HL=3
  • apache-camel-2.13.0 CD=98 UD=11 HL=3
  • apache-camel-2.14.0 CD=98 UD=11 HL=3
  • apache-camel-2.15.0 CD=98 UD=11 HL=3

Apache Cxf

  • apache-cxf-2.0.6 CD=110 UD=52 HL=4
  • apache-cxf-2.1.1 CD=132 UD=61 HL=4
  • apache-cxf-2.2.1 CD=150 UD=74 HL=3
  • apache-cxf-2.3.0 CD=197 UD=79 HL=3
  • apache-cxf-2.4.0 CD=199 UD=86 HL=4
  • apache-cxf-2.5.0 CD=206 UD=88 HL=3
  • apache-cxf-2.6.0 CD=146 UD=92 HL=3
  • apache-cxf-2.7.0 CD=196 UD=95 HL=3

Hibernate

  • hibernate-core-4.0.0.Final CD=153 UD=30 HL=5
  • hibernate-core-4.1.0.Final CD=163 UD=33 HL=5
  • hibernate-core-4.2.0.Final CD=171 UD=32 HL=6
  • hibernate-core-4.3.0.Final CD=230 UD=62 HL=6

Tools

The Arcan tool used for detecting the architectural smells is available here: http://essere.disco.unimib.it/wiki/arcan

  • The SALib library (Python) for running the sensitivity analysis on input parameters (i.e., smells or packages) is available here: https://github.com/SALib/SALib

This is a quick tutorial on how to run SALib to process the sensitivity files above:

The objective values (or model values) for the sensitivity analysis are obtained with the ADI formula:

ADI Formula

Clone this wiki locally