Scalarm - a platform for data farming
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
deployment_scripts
docs/adapters
quick_install
util_scripts
CHANGELOG
README.md
Vagrantfile
env.source.example
start.sh
stop.sh

README.md

Scalarm Logo

Scalarm is a platform supporting scientific experiments which follow parameter studies and data farming methodologies - meaning they are based on execution of the same application multiple times with different input parameters to explore how it behaves in different conditions.

The following steps of such experiments are supported:

  • input parameter space specification with design of experiment methods,
  • application/simulation model execution on different computing resources like clouds, grids, computer clusters and remotely accessible servers,
  • data collection and analysis.

Quick start

Tested on Ubuntu 14.04 and Ubuntu 16.04

Notice: sudo permissions are required, since some system packages will be installed and port 443 is used.

If you prefer a video tutorial it is available at - https://youtu.be/g84QlUK_Z0s

Handy oneliner to install Scalarm with graphical user interface (GUI) exposed on 192.168.0.1 :

PUBLIC_NGINX_ADDRESS=192.168.0.1 bash -c '\curl -sSL https://raw.githubusercontent.com/Scalarm/scalarm/master/quick_install/local_nginx_paths.sh | bash'

If you want to use Scalarm git branch other than default master, set a GIT_BRANCH env before installation:

export GIT_BRANCH=development

Data farming process

The data farming methodology focuses on data generation and analysis with computer simulation and high performance computing (HPC) systems. The whole process is as follows:

Data farming

Architecture

The Scalarm platform consits of several loosely coupled services. It's architecture follows the master-worker design pattern, where the master part is responsible for experiment coordination, while the worker part handles actual computations. In addition, Scalarm services can be run on different infrastructures, hence its actual runtime deployment can be divided vertically and horizontally as depicted below, where the amount of resources for each part can be dynamically adjusted to the actual load.

Scalarm architecture

More information

Scalarm is a fully open source platform developed at AGH University of Science and Technology, Krakow, Poland by people from Department of Computer Science and Academic Computer Center Cyfronet.

More information about the platform can be found on:

We are always looking for collaborators and users :)

License

MIT