MONDO-SAM: A Framework to Systematically Assess MDE Scalability
MONDO-SAM provides a common framework for benchmark developers written in Java, Python and R.
- Java SE 1.6
- Python 3 (tested with 3.4)
- R (3.2+)
- Maven 3
To import the projects to Eclipse, use the following command to generate the
Set the value of the
M2_REPO variable in your Eclipse workspace. There are multiple ways to achieve this:
- Use the
ECLIPSE_WORKSPACE=/path/to/your/workspace mvn eclipse:configure-workspace -Declipse.workspace="$ECLIPSE_WORKSPACE"
Go to Window | Preferences, Java | Build Path | Classpath Variables and add the
M2_REPOvariable. It should point to the absolute path of the
Installing the m2e Eclipse plug-in also sets the value of
M2_REPOautomatically. However, note that m2e is not compatible with projects generated by the
eclipse:eclipsecommand so you should not convert these projects to Maven projects in Eclipse.
The first step is to define the own scenario and phase implementations. For further information, read the wiki.
Every benchmark result is saved to a JSON file. In order to convert the results to a CSV file, run the following:
To add specific paths, use the optional arguments. For more information, see
The operation above is also necessary for the reporting mechanism.
Reporting in R
There is a chance to visualize the measurement results on diagrams. At first, install R and the required packages:
Install R 3.2+. On Ubuntu 15.10+, this is as simple as issuing:
sudo apt-get install -y r-base r-base-dev cd reporting sudo R -f install.R
On older systems, follow the Ubuntu Packages for R guide.
- Download and install R
- Adjust the system environment variables
- In a command prompt with administrator rights, run:
R -f install.R
After converting the results to the right CSV format (see above), it becomes feasible to start the reporting. The first option is that to use the interactive application.
Besides the primary dependencies, it is necessary to install additional R packages:
cd reporting/shiny sudo R -f install.R
In order to launch the application, run the following script from the
It is possible to run the application in terminal (from the
reporting directory again):
R -e "shiny::runApp('./shiny/', launch.browser=TRUE)"
To run the application from a different source, just change the path parameter (
Or, import the
reporting/shiny/server.R file into RStudio and click on the
Run App button. However, note that some features are not available in RStudio, for example, it is not feasible to export the configurations.
The second option is that to run the plot generation scripts directly without interactivity. In this case, it is necessary to use an external configuration file (
At first, adjust the reporting configuration settings in the
reporting/config.json file. These parameters determine exactly what kind of plots will be generated. For further information, read the reporting section on wiki.
Finally, it is already possible to generate the diagrams. The easiest way for this to use the Python wrapper module:
./reporting/report.py --source source_csv_path --output output_folder_path --config config_json_path`
Insert valid parameters instead the paths. The wrapper module actually invokes the
reporting/generate_diagrams.R script, however it is still recommended to use it for two reasons. The Python script:
- solves the working directory problem and
- provides a more useable argument parser than R.
It is possible as well to export the configurations that are adjusted in the interactive interface, and reuse it as an external configuration for the