Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Eclipse Feature Location Benchmark for Software Families uses variants of Eclipse from which a ground-truth is extracted.
A set of Feature Location techniques have been already integrated in BUT4Reuse.
Do you have your own feature location technique?
Do you want to compare it with others in this benchmark?
Instructions on integrating your feature location technique to be used within EFLBench:
Cite as: Jabier Martinez, Tewfik Ziadi, Mike Papadakis, Tegawendé F. Bissyandé, Jacques Klein, and Yves Le Traon : Feature Location Benchmark for Software Families using Eclipse Community Releases, ICSR 2016, Limassol, Cyprus, 5-7 June
Get the Eclipse package variants
Select the distribution from which you want to create the benchmark: http://www.eclipse.org/downloads/packages/all
For example, you can use Kepler SR2, download the win 64 bits. http://www.eclipse.org/downloads/packages/release/Kepler/SR2
Notice that it does not matter that you are not using windows nor 64 bits. You will not need to execute it and EFLBench automatic extractor will not need to execute them to prepare the feature location benchmark.
Download and Unzip all. This process can take some time depending on your internet connection and computer. At the end, you should have a folder like the one in the image with the 12 packages of the Kepler SR2 release
Install BUT4Reuse and launch it. For this follow the instructions at [Installation](Installation) Install BUT4Reuse and prepare EFLBench
Create a project and an artefact model. For example, call the project packagesKepler and call kepler to the artefactmodel. Then open the kepler.artefactmodel in the artefactmodel editor by double clicking it. Select the root and in the properties view, put kepler as name. For these steps you can find further information in the BUT4Reuse user manual: UserManual
Be sure that the kepler.artefactmodel is still open in the editor.
Drag and Drop the unzipped folders of the eclipse distributions in the BUT4Reuse's Input Drop view.
Select each of the artefacts and in the Properties view, be sure that the URI is pointing to the eclipse folder. Most probably you will need to add eclipse/ manually. For example file:/C:/keplerPackages/eclipse-jee-kepler-SR1-win32-x86_64/eclipse/
Right click the artefact model and click on "Create Eclipse Feature Location Benchmark" and wait. Launch EFLBench automatic extractor
This action can take around 5 minutes depending on your computer. Once it is finished a summary is shown.
A benchmark folder is created
actualFeatures folder: This contains the ground truth. Inside the folder you will find a txt file for each feature and in these txt files you have, in each line, the ids of the plugins that constitutes this feature.
eclipse_kepler.featurelist: This contains the information to launch feature location techniques. You can open it with the featurelist editor. It is a feature list with feature names and descriptions. In the Eclipse properties view, by selecting a feature you can see in which artefacts this feature is implemented.
Then we have two graphml files with mined information that can be exploited by feature location techniques. You can use Gephi for visualising graphml files: https://gephi.org/ . If you want to include a feature location technique that leverages this information you can use for example the blueprints library, com.tinkerpop.blueprints.Graph. However, plugin element dependencies to other plugin elements can be obtained through the getDependencies method of the Element interface
actualFeatures.graphml: a graph containing feature dependencies
pluginDependencies.graphml: a graph containing plugin dependencies
The following figure shows the feature list with Eclipse CVS Client feature selected and their properties name, description and in which packages it is present.
To change the feature location techniques go to Window -> Preferences, and select BUT4Reuse Benchmark usage
Here you can find the feature location techniques registered.
In the preferences, in Block creation, you can select for example Formal Concept Analysis (select only one).
Then select also only one of the feature location techniques, for example SFS
In order to reduce the time that you will have to wait until getting the results, in the preferences, in Visualisation, only check Metrics and Eclipse benchmark specific visualisation. Also in Constraints discovery, do not select any of the techniques.
Metrics visualisation will provide us the information about the time performance and the other will provide the precision and recall results.
Right click the root of the Feature list and click on Feature location
They will ask you for the adapter to use, the abstraction phase for the variants. Select only the Eclipse4Benchmark adapter
A csv file (comma-separated-value) called resultsPrecisionRecall_techniqueSelected_timestamp will be created containing the precision and recall for each feature
The time performance is shown in the Metrics view. It is at the end of metrics report.