Skip to content

Commit

Permalink
Merge branch 'master' of github.com:wrench-project/wrench
Browse files Browse the repository at this point in the history
  • Loading branch information
henricasanova committed Nov 23, 2017
2 parents f781c29 + 500374c commit 5cfb0db
Show file tree
Hide file tree
Showing 11 changed files with 217 additions and 18 deletions.
23 changes: 13 additions & 10 deletions README.md
Expand Up @@ -3,6 +3,7 @@
[![License: LGPL v3][license-badge]](LICENSE.md)
[![Coverage Status][coveralls-badge]][coveralls-link]
[![Codacy Badge][codacy-badge]][codacy-link]
[![SonarCloud Badge][sonarcloud-badge]][sonarcloud-link]

<img src="doc/images/logo-vertical.png" width="100" />

Expand Down Expand Up @@ -42,17 +43,19 @@ If all dependencies are installed, compiling and installing WRENCH is as simple
```bash
cmake .
make
make install
make install # try "sudo make install" if you don't have the permission to write
```



[travis-badge]: https://travis-ci.org/wrench-project/wrench.svg?branch=master
[travis-link]: https://travis-ci.org/wrench-project/wrench
[license-badge]: https://img.shields.io/badge/License-LGPL%20v3-blue.svg
[coveralls-badge]: https://coveralls.io/repos/github/wrench-project/wrench/badge.svg?branch=master
[coveralls-link]: https://coveralls.io/github/wrench-project/wrench?branch=master
[release-badge]: https://img.shields.io/github/release/wrench-project/wrench.svg
[release-link]: https://github.com/wrench-project/wrench/releases
[codacy-badge]: https://img.shields.io/codacy/grade/aef324ea84474fff979a8ff19a4e4681.svg
[codacy-link]: https://www.codacy.com/app/WRENCH/wrench?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=wrench-project/wrench&amp;utm_campaign=Badge_Grade
[travis-badge]: https://travis-ci.org/wrench-project/wrench.svg?branch=master
[travis-link]: https://travis-ci.org/wrench-project/wrench
[license-badge]: https://img.shields.io/badge/License-LGPL%20v3-blue.svg
[coveralls-badge]: https://coveralls.io/repos/github/wrench-project/wrench/badge.svg?branch=master
[coveralls-link]: https://coveralls.io/github/wrench-project/wrench?branch=master
[release-badge]: https://img.shields.io/github/release/wrench-project/wrench.svg
[release-link]: https://github.com/wrench-project/wrench/releases
[codacy-badge]: https://img.shields.io/codacy/grade/aef324ea84474fff979a8ff19a4e4681.svg
[codacy-link]: https://www.codacy.com/app/WRENCH/wrench?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=wrench-project/wrench&amp;utm_campaign=Badge_Grade
[sonarcloud-badge]: https://sonarcloud.io/api/badges/measure?key=wrench&metric=ncloc
[sonarcloud-link]: https://sonarcloud.io/dashboard?id=wrench
41 changes: 41 additions & 0 deletions doc/getting_started.md
Expand Up @@ -10,6 +10,47 @@ Getting Started {#getting-started}
The first step is to have WRENCH library installed. If it is not done yet, please
follow the instructions to [install it](@ref install).

# Running a First Example # {#getting-started-example}

WRENCH distribution provides an example WMS implementation (`simple-wms`) available
in the `examples` folder. A simple installation via `make && make install` already
compiles all examples.

WRENCH provides two implementations for the `simple-wms` example: a cloud-based
implementation `wrench-simple-wms-cloud`, and an implementation to run workflows
in a batch system (e.g., SLURM) `wrench-simple-wms-batch`.

A successful WRENCH installation will put the examples binaries in the `/usr/local/bin`
folder (for MacOS and most Linux distributions). To run the examples, simply use
one of the following commands:

~~~~~~~~~~~~~{.sh}
# running the cloud-based implementation
wrench-simple-wms-cloud <PATH-TO-WRENCH-SRC-FOLDER>/examples/two_hosts.xml <PATH-TO-WRENCH-SRC-FOLDER>/examples/genome.dax
# running the batch-based implementation
wrench-simple-wms-batch <PATH-TO-WRENCH-SRC-FOLDER>/examples/two_hosts.xml <PATH-TO-WRENCH-SRC-FOLDER>/examples/genome.dax
~~~~~~~~~~~~~

## Understanding the Simple-WMS Examples {#getting-started-example-simplewms}

The `simple-wms` example requires two arguments: (1) a [SimGrid virtual platform
description file](http://simgrid.gforge.inria.fr/simgrid/3.17/doc/platform.html); and
(2) a WRENCH workflow file.

_**SimGrid virtual platform description file:**_
In [SimGrid](http://simgrid.gforge.inria.fr), any study must entail the description
of the platform on which you want to simulate your application. This file includes
definitions of computing hosts, clusters, storage, network links and routes, etc.
A detailed description on how to build your platform description file can be found
[here](http://simgrid.gforge.inria.fr/simgrid/3.17/doc/platform.html).

_**WRENCH workflow file:**_
WRENCH provides native parsers for [DAX](http://workflowarchive.org) (DAG in XML)
and [JSON](http://workflowhub.org/traces/) formats. Please, refer to the hyperlinked
texts for their respective documentations.


# Preparing the Environment # {#getting-started-prep}

@WRENCHNotInternalDoc
Expand Down
4 changes: 2 additions & 2 deletions doc/index.md
Expand Up @@ -37,11 +37,11 @@ Through the use of SimGrid, WRENCH provides the ability to:

# Architecture # {#overview-architecture}

WRENCH is an _open-source library_ for deloping simulators. It is neither a graphical
WRENCH is an _open-source library_ for developing simulators. It is neither a graphical
interface nor a stand-alone simulator. WRENCH exposes several high-level simulation
abstractions to provide the **building blocks** for developing custom simulators.

WRENCH comprises four disctinct modules, each designed as a simulation component:
WRENCH comprises four distinct modules, each designed as a simulation component:

- **Simulation Engine:** the simulation code that provides the necessary models to simulate arbitrarily interconnected hardware resources (compute, network, and storage).
- **Compute Services:** abstractions for the simulated infrastructure components that can execute workflow tasks (e.g., bare-metal servers, cloud platforms, batch-scheduled clusters, etc.).
Expand Down
2 changes: 1 addition & 1 deletion doc/install.md
Expand Up @@ -43,7 +43,7 @@ tar xf @WRENCHRelease.tar.gz
cd @WRENCHRelease
cmake .
make
make install
make install # try "sudo make install" if you don't have the permission to write
~~~~~~~~~~~~~

If you want to stay on the bleeding edge, you should get the latest git version, and recompile it as you would do for an official archive:
Expand Down
2 changes: 2 additions & 0 deletions doc/layout/customdoxygen.css
Expand Up @@ -511,13 +511,15 @@ span.SRScope {
.contents h1 {
font-size: 22px;
font-weight: 700 !important;
padding-top: 30px;
padding-bottom: 5px;
border-bottom: 1px solid #ddd;
}

.contents h2 {
font-size: 18px;
font-weight: 700 !important;
padding-top: 15px;
}

.contents h3 {
Expand Down
File renamed without changes.
File renamed without changes.
15 changes: 11 additions & 4 deletions examples/simple-wms/CMakeLists.txt
Expand Up @@ -17,9 +17,16 @@ set(SOURCE_FILES
scheduler/MaxMinScheduler.cpp
scheduler/pilot_job/CriticalPathScheduler.h
scheduler/pilot_job/CriticalPathScheduler.cpp
main.cpp
)

add_executable(simple-wms ${SOURCE_FILES})
target_link_libraries(simple-wms wrench ${SIMGRID_LIBRARY} ${PUGIXML_LIBRARY} ${LEMON_LIBRARY})
install(TARGETS simple-wms DESTINATION bin)
# cloud example
set(APP_CLOUD_FILES SimpleWMSCloud.cpp)
add_executable(wrench-simple-wms-cloud ${SOURCE_FILES} ${APP_CLOUD_FILES})
target_link_libraries(wrench-simple-wms-cloud wrench ${SIMGRID_LIBRARY} ${PUGIXML_LIBRARY} ${LEMON_LIBRARY})
install(TARGETS wrench-simple-wms-cloud DESTINATION bin)

# batch example
set(APP_BATCH_FILES SimpleWMSBatch.cpp)
add_executable(wrench-simple-wms-batch ${SOURCE_FILES} ${APP_BATCH_FILES})
target_link_libraries(wrench-simple-wms-batch wrench ${SIMGRID_LIBRARY} ${PUGIXML_LIBRARY} ${LEMON_LIBRARY})
install(TARGETS wrench-simple-wms-batch DESTINATION bin)
File renamed without changes.
146 changes: 146 additions & 0 deletions examples/simple-wms/SimpleWMSCloud.cpp
@@ -0,0 +1,146 @@
/**
* Copyright (c) 2017. The WRENCH Team.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*/

#include <iostream>
#include <wrench.h>

#include "SimpleWMS.h"
#include "scheduler/RandomScheduler.h"
#include "scheduler/CloudScheduler.h"

int main(int argc, char **argv) {

wrench::Simulation simulation;

simulation.init(&argc, argv);

if (argc != 3) {
std::cerr << "Usage: " << argv[0] << " <xml platform file> <workflow file>" << std::endl;
exit(1);
}

char *platform_file = argv[1];
char *workflow_file = argv[2];

// reading and parsing the workflow file
std::cerr << "Loading workflow..." << std::endl;
wrench::Workflow workflow;
wrench::WorkflowUtil::loadFromDAX(workflow_file, &workflow);

std::cerr << "The workflow has " << workflow.getNumberOfTasks() << " tasks " << std::endl;
std::cerr.flush();

// reading and parsing the SimGrid virtual platform description file
std::cerr << "Instantiating SimGrid platform..." << std::endl;
simulation.instantiatePlatform(platform_file);

// obtaining the list of hosts
std::vector<std::string> hostname_list = simulation.getHostnameList();

std::string storage_host = hostname_list[(hostname_list.size() > 3) ? 2 : 1];

std::cerr << "Instantiating a SimpleStorageService on " << storage_host << "..." << std::endl;

wrench::StorageService *storage_service = simulation.add(std::unique_ptr<wrench::SimpleStorageService>(
new wrench::SimpleStorageService(storage_host, 10000000000000.0)));
// wrench::StorageService *simulation = simulation.add(
// std::unique_ptr<wrench::SimpleStorageService>(new wrench::SimpleStorageService(storage_host, 10.0)));

std::string wms_host = hostname_list[0];


std::string executor_host = hostname_list[(hostname_list.size() > 1) ? 1 : 0];

wrench::ComputeService *cloud_service = new wrench::CloudService(
wms_host, true, true, storage_service,
{{wrench::CloudServiceProperty::STOP_DAEMON_MESSAGE_PAYLOAD, "666"}});
std::vector<std::string> execution_hosts = {executor_host};

try {

std::cerr << "Instantiating a MultiCore Job executor on " << executor_host << "..." << std::endl;
// simulation.add(
// std::unique_ptr<wrench::MultihostMulticoreComputeService>(
// new wrench::MultihostMulticoreComputeService(executor_host, true, true,
// storage_service,
// {{wrench::MultihostMulticoreComputeServiceProperty::STOP_DAEMON_MESSAGE_PAYLOAD, "666"}})));

simulation.add(std::unique_ptr<wrench::ComputeService>(cloud_service));

// std::cerr << "Instantiating a MultiCore Job executor on " << executor_host << "..." << std::endl;
// simulation.add(std::unique_ptr<wrench::MultihostMulticoreComputeService>(
// new wrench::MultihostMulticoreComputeService(executor_host, true, false,
// {{wrench::MultihostMulticoreComputeService::Property::STOP_DAEMON_MESSAGE_PAYLOAD, "666"}})));

// std::cerr << "Instantiating a MultiCore Job executor on " << exexutor_host << "..." << std::endl;
// simulation.add(std::unique_ptr<wrench::MultihostMulticoreComputeService>(
// new wrench::MultihostMulticoreComputeService(executor_host, false, true,
// {{wrench::MultihostMulticoreComputeService::Property::STOP_DAEMON_MESSAGE_PAYLOAD, "666"}})));

// std::cerr << "Instantiating a MultiCore Job executor on " << exexutor_host << "..." << std::endl;
// simulation.add(std::unique_ptr<wrench::MultihostMulticoreComputeService>(
// new wrench::MultihostMulticoreComputeService(executor_host, true, true,
// {{wrench::MultihostMulticoreComputeService::Property::STOP_DAEMON_MESSAGE_PAYLOAD, "666"}})));

} catch (std::invalid_argument &e) {

std::cerr << "Error: " << e.what() << std::endl;
std::exit(1);

}

std::cerr << "Instantiating a WMS on " << wms_host << "..." << std::endl;

// WMS Configuration
wrench::WMS *wms = simulation.setWMS(
std::unique_ptr<wrench::WMS>(
new wrench::SimpleWMS(&workflow,
std::unique_ptr<wrench::Scheduler>(
// new wrench::RandomScheduler()),
new wrench::CloudScheduler(cloud_service, execution_hosts,
&simulation)),
wms_host)));

// wms->setPilotJobScheduler(std::unique_ptr<wrench::PilotJobScheduler>(new wrench::CriticalPathScheduler()));

// wms->addStaticOptimization(std::unique_ptr<wrench::StaticOptimization>(new wrench::SimplePipelineClustering()));
// wms->addDynamicOptimization(std::unique_ptr<wrench::DynamicOptimization>(new wrench::FailureDynamicClustering()));

std::string file_registry_service_host = hostname_list[(hostname_list.size() > 2) ? 1 : 0];

std::cerr << "Instantiating a FileRegistryService on " << file_registry_service_host << "..." << std::endl;
std::unique_ptr<wrench::FileRegistryService> file_registry_service(
new wrench::FileRegistryService(file_registry_service_host));
simulation.setFileRegistryService(std::move(file_registry_service));

std::cerr << "Staging input files..." << std::endl;
std::set<wrench::WorkflowFile *> input_files = workflow.getInputFiles();
try {
simulation.stageFiles(input_files, storage_service);
} catch (std::runtime_error &e) {
std::cerr << "Exception: " << e.what() << std::endl;
return 0;
}

std::cerr << "Launching the Simulation..." << std::endl;
try {
simulation.launch();
} catch (std::runtime_error &e) {
std::cerr << "Exception: " << e.what() << std::endl;
return 0;
}
std::cerr << "Simulation done!" << std::endl;

std::vector<wrench::SimulationTimestamp<wrench::SimulationTimestampTaskCompletion> *> trace;
trace = simulation.output.getTrace<wrench::SimulationTimestampTaskCompletion>();
std::cerr << "Number of entries in TaskCompletion trace: " << trace.size() << std::endl;
std::cerr << "Task in first trace entry: " << trace[0]->getContent()->getTask()->getId() << std::endl;

return 0;
}
2 changes: 1 addition & 1 deletion sonar-project.properties
@@ -1,6 +1,6 @@
sonar.projectKey=wrench
sonar.projectName=WRENCH
sonar.projectVersion=1.0.0-dev
sonar.projectVersion=1.0-alpha

sonar.links.homepage=http://wrench-project.org
sonar.links.ci=https://travis-ci.org/wrench-project/wrench
Expand Down

0 comments on commit 5cfb0db

Please sign in to comment.