HOME

Welcome

The STARS framework provides a set of tools to enable the rigorous statistical analysis of stochastic event-based simulations. It can be run in a number of different deployment configurations, namely: grid, cluster, or stand-alone. This enables experiments of varying scales to be conducted using STARS. Within the framework experiments can be broken down into three main components:

1. Simulation Model
2. Analysis Engine
3. Experiment Automation

The simulation model is the first component and is problem specific and user supplied. The STARS framework was developed around a MANET model constructed using the OMNeT++ simulator; however, the design allows for other simulators to be used. Instrumentation is provided by the STARS framework to measure features of the model for analysis. This custom instrumentation is used for two reasons. First it provides a common data format so results can be shared across simulators. Second it is designed to be space and time efficient to support large data sets.

The analysis engine is a component of the STARS framework. It is written in MATLAB and supports the Distributed Computing Server to provide speedup on larger data sets. The analysis engine addresses a number of statistical issues which come into play when working with stochastic data, namely: start-up transients, and averaging over ergodic data. Testing for stationarity in the feature allows the estimated empirical distribution to be calculated. By comparing these distributions to one another the ergodic samples can be grouped together and averaged. This can expose different modes of behaviour for the feature.

The experiment automation component of the STARS framework enables hands-free operation of the simulation model and analysis engine components. It makes use of Python, using the pyMPI interpreter, to allow custom definition of experiment processes. These experiment processes are created based on two inputs: a resource package, and a workfile. The former contains the simulation model, analysis engine, and any other resources. The workfile defines the experiment and contains the model and analysis configuration. This allows a single resource package to be used by multiple workfiles to create new experiments. Currently the automation is supported for cluster and stand-alone operation only, support for grid resources is planned for future releases.

For additional information please visit the List of Publications that use STARS

Getting Started

It is recommended that new users of the STARS framework follow the list of steps below. These will describe the download, setup, and testing of the STARS framework prior to integrating it with your simulation. Once the framework is installed and working the simulated model can be instrumented. The analysis can then be configured to report on features contained within the data recorded about the model. Finally, the experiment process can be automated to enable hands-free operation.

1. Downloading the STARS Framework
2. Setup and configuration
3. Testing the installation
4. Using the instrumentation
5. Analyzing features of the model
6. Automating experiments
7. Running Experiments
8. Analysis Results