Skip to content
ECHO: An Adaptive Orchestration Platform for Hybrid Dataflows across Cloud and Edge Resources
Java Python Other
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
echo_master_service
echo_platform_service
input-dags
misc
.gitignore
Dockerfile
LICENSE.txt
README.md

README.md

ECHO

Introduction

ECHO (Orchestration platform for Hybrid dataflows across Cloud and Edge) is a platform that operates across diverse distributed resources to enable execution of complex analytical dataflows. It can operate on diverse data models - streams, micro-batches and files, and interface with native runtime engines like TensorFlow and Sorm to execute them. It manages the applcation's lifecycle, and can schedule the dataflow on Edge, Fog and Cloud resources while enabling dynamic task migration between the resources.

Publication: ECHO: An Adaptive Orchestration Platform for Hybrid Dataflows across Cloud and Edge, Pushkara Ravindra, Aakash Khochare, Siva Prakash Reddy, Sarthak Sharma, Prateeksha Varshney, Yogesh Simmhan, 15th International Conference on Service-Oriented Computing (ICSOC) 2017, Malaga, Spain, https://doi.org/10.1007/978-3-319-69035-3_28 and https://arxiv.org/abs/1707.00889

Dataflows

There are multiple example dataflows that are present in the input-dags folder.

Design

The Framework consists of mainly two modules

  • ECHO Master Service - This is the central command module that manages all the resources and the applications that make up the framework. It is the point of contact for any user wishing to deploy a new Application onto the platform, or to register a new edge device into the platform

  • ECHO Platform Service - This is the module that runs on every individual edge device. It maintains a link with the Master Service and executes the control signals received from the master.

Resource Directory

The Resource Directory is present in the Registry module. The target folder will contain HyperCatServer.war which should be placed in the ROOT directory of a tomcat server. The Resource Directory can then be accessed as <ip_of_tomcat_server>:8080/cat

MQTT Broker

The platform uses MQTT messages to transmit control signals between the Master Service and the Platform Service. ECHO uses mosquitto MQTT service, which must be installed on the server running the Master Service. The service can be installed via apt using sudo apt-get install mosquitto

Usage

Submit Application to the master by sending a POST request to the <ip_of_cloud_master>:8099/DAG with the DAG representing the application as a request body.

Please refer the paper for detailed information about the platform - https://arxiv.org/abs/1707.00889

You can’t perform that action at this time.