Tools used for the evaluation of RefactoringMiner 2.0
We used the >7000 refactorings in this oracle as the ground truth for evaluating the precision and recall of the 6 tools.
We made the following updates to this tool:
- Updated the description format of the reported refactorings to match that of the oracle
- Updated the Git checkout process: ** if git checkout for a commit fails, try 'git checkout master -f' and then try to checkout the commit again ** If this does not work, we use the Github API to download the zip of the project at the particular commit
- Added Extract And Move method feature by : ** If the 'from' and the 'to' methods belong to different classes, we classify it as Extract And Move Method Refactoring ** Ensuring that no rename, move and move & rename class refactoring led to this conclusion
To evaluate this tool over the oracle:
- Download the latest JSON from the oracle and update this file.
- Fill in your GitHub credentials here
- Provide a path to the folder where repositories could be cloned or found, here
- Execute TestAllRefactoringsRD.testAllRefactorings to run the evaluation
- Output of the evaluation will be found in the folder named 'Output' in the working directory.
- Execute the main method in AnalyseResults to calculate precision and recall per refactoring.
We made the following updates to this tool:
- Updated the description format of the reported refactorings to match that of the oracle
- Updated the Git checkout process: ** if git checkout for a commit fails, try 'git checkout master -f' and then try to checkout the commit again ** If this does not work, we use the Github API to download the zip of the project at the particular commit
- Added Extract And Move method feature by : ** If the 'from' and the 'to' methods belong to different classes, we classify it as Extract And Move Method Refactoring ** Ensuring that no rename, move and move & rename class refactoring led to this conclusion
To evaluate this tool over the oracle:
- Download the latest JSON from the oracle and update this file.
- Fill in your GitHub credentials here
- Provide a path to the folder where repositories could be cloned or found, here
- Execute TestAllRefactoringsRD.testAllRefactorings to run the evaluation
- Output of the evaluation will be found in the folder named 'Output' in the working directory.
- Execute the main method in AnalyseResults to calculate precision and recall per refactoring.
We made the following updates to this tool:
- Updated the description format of the reported refactorings to match that of the oracle
- Updated the tool to not report Move and/or Rename method refactorings when the method itself is abstract
To evaluate this tool over the oracle:
- Download the latest JSON from the oracle and update this file.
- Since this version of RefDiff does not require the entire repository, it fetches the changed files from this folder
- Execute TestAllRefactoringsRD.testAllRefactorings to run the evaluation
- Output of the evaluation will be found in the folder named 'Output' in the working directory.
- Execute the main method in AnalyseResults to calculate precision and recall per refactoring.
We made the following updates to this tool:
- Updated the description format of the reported refactorings to match that of the oracle
- Updated the tool to not checkout each commit, instead query the changes from using the Github API
To evaluate this tool over the oracle:
- Download the latest JSON from the oracle and update this file.
- Execute TestAllRefactorings.testAllRefactorings to run the evaluation
- Output of the evaluation will be found in the folder named 'Output' in the working directory.
- Execute the main method in AnalyseResults to calculate precision and recall per refactoring.
To evaluate RefactoringMiner 2.0:
- Get the latest version of it from here
- Follow steps similar to RefactoringMiner 1.0
GumTree is used for the evaluation of Rename and Change Type refactoring. To evaluate GumTree 2.1.2:
- Download the latest JSON from the oracle and update this file.
- Execute TestAllRefactorings.testAllRefactorings to run the evaluation
- Output of the evaluation will be found in the folder named 'Output' in the working directory.
- Execute the main method in AnalyseResults to calculate precision and recall per refactoring.
