Reporting tools for the MONDO-SAM framework
R Shell
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
converter
diagrams
.gitignore
.travis.yml
LICENSE
README.md
clear-diagrams.sh
init-sample.sh
install.R
process.R
report.R
report.sh
reporting.Rproj
results.csv
sample-results.csv
test.R

README.md

MONDO-SAM reporting

Build Status

Reporting tools for MONDO-SAM

Prerequisites

Install R and the required packages.

Linux

Install R 3.2+. On Ubuntu 15.10+, this is as simple as issuing:

sudo apt-get install -y r-base r-base-dev
sudo R -f install.R

On older Ubuntu systems, follow the Ubuntu Packages for R guide. For other Linux systems, refer to the CRAN Linux pages.

Windows

Install R and adjust the system environment variables.

In a command prompt with administrator rights, run:

R -f install.R

To edit the script, it is recommended to use RStudio.

Usage

To draw a few simple plots, issue the following command:

./report.sh

The results should be the results.csv file. For a sample CSV file, open the sample-results.csv file with your favorite spreadsheet software or text editor.

The schema attributes are defined in the following list. For each attribute, we list an example from the Train Benchmark framework πŸš‚.

  • Scenario
    • defines the workflow of the benchmark
    • πŸš‚ the Batch scenario defines a workflow with a single read and query operation.
  • Tool
    • defines the system under benchmark with it's custom settings
    • πŸš‚ EMF-IncQuery (Local Search)
  • Run
    • specifies the index of the run
    • πŸš‚ there are usually 5 runs, so the Run attribute takes a value between 1 and 5
  • Case
    • defines the benchmark case
    • πŸš‚ the query PosLength
  • Artifact
    • defines the artifact which used by the benchmark
    • πŸš‚ the model size used by the queries and transformations.
  • Phase
    • defines the phase of the benchmark
    • πŸš‚ the Read phase loads the model
  • Metric
    • the metric stored by the particular row
    • πŸš‚ the benchmark uses two metrics; Time (for measuring the execution time in nanoseconds) and Matches
  • Iteration
    • defines the index of the current iteration if a step is executed multiple times
    • πŸš‚ for the Transformation and Recheck phases in the Repair scenario, the Iteration attribute takes a value between 1 and 10
  • Value
    • The value of the metric.
    • πŸš‚ for Time, a valid value is 456123123 (for an approximately half-second operation), for Matches, a valid value is 5

Related projects