Spring Cloud Data Flow provides orchestration for data microservices, including both stream and task processing
Java XSLT Other
Permalink
Failed to load latest commit information.
.mvn Update maven wrapper to mvn 3.3.9 Jan 4, 2017
spring-cloud-dataflow-completion [artifactory-release] Next development version Jan 18, 2017
spring-cloud-dataflow-configuration-metadata [artifactory-release] Next development version Jan 18, 2017
spring-cloud-dataflow-core [artifactory-release] Next development version Jan 18, 2017
spring-cloud-dataflow-dependencies [artifactory-release] Next development version Jan 18, 2017
spring-cloud-dataflow-docs [artifactory-release] Next development version Jan 18, 2017
spring-cloud-dataflow-registry [artifactory-release] Next development version Jan 18, 2017
spring-cloud-dataflow-rest-client [artifactory-release] Next development version Jan 18, 2017
spring-cloud-dataflow-rest-resource [artifactory-release] Next development version Jan 18, 2017
spring-cloud-dataflow-server-core [artifactory-release] Next development version Jan 18, 2017
spring-cloud-dataflow-server-local-autoconfig [artifactory-release] Next development version Jan 18, 2017
spring-cloud-dataflow-server-local [artifactory-release] Next development version Jan 18, 2017
spring-cloud-dataflow-shell-core [artifactory-release] Next development version Jan 18, 2017
spring-cloud-dataflow-shell [artifactory-release] Next development version Jan 18, 2017
spring-cloud-starter-dataflow-server-local [artifactory-release] Next development version Jan 18, 2017
.gitignore gh-496 Add support to relaunch and stop Batch Jobs Apr 22, 2016
.settings.xml Fix typo in .settings.xml Feb 11, 2016
.travis.yml Update travis integration to build non PR builds Dec 18, 2015
CODE_OF_CONDUCT.adoc Add Code of Conduct Apr 27, 2016
LICENSE maven infrastructure Jul 21, 2015
README.adoc Updates to README and reference guide with recent changes Nov 17, 2016
mvnw GH-163, GH-173, GH-215 Add travis, mvnw, and update README Dec 8, 2015
mvnw.cmd #642 Maven Wrapper cmd script issue fixed Jun 3, 2016
pom.xml [artifactory-release] Next development version Jan 18, 2017

README.adoc

Latest Release Version Latest Snapshot Version

Spring Cloud Data Flow Build Status Stories Ready Stories In Progress

The Spring Cloud Data Flow project provides orchestration for data microservices, including long-lived Spring Cloud Stream applications and short-lived Spring Cloud Task applications.

Components

The Core domain module includes the concept of a stream that is a composition of spring-cloud-stream modules in a linear pipeline from a source to a sink, optionally including processor module(s) in between. The domain also includes the concept of a task, which may be any process that does not run indefinitely, including Spring Batch jobs.

The App Registry maintains the set of available apps, and their mappings to URIs. For example, if relying on Maven coordinates, an app’s URI would be of the format: maven://<groupId>:<artifactId>:<version>

The Data Flow Server is a Spring Boot application that provides a common REST API and UI. For each runtime environment there is a different version of the Data Flow Server that depends upon a deployer SPI implementation for that environment. The github locations for these Data Flow Servers are:

There are also community maintained Spring Cloud Data Flow implementations for Hashicorp’s Nomad and RedHat’s Openshift.

The deployer SPI mentioned above is defined within the Spring Cloud Deployer project. That provides an abstraction layer for deploying the apps of a given stream or task and managing their lifecycle. The github locations for the corresponding Spring Cloud Deployer SPI implementations are:

The Shell connects to the Data Flow Server’s REST API and supports a DSL that simplifies the process of defining a stream or task and managing its lifecycle.

Instructions for running the Data Flow Server for each runtime environment can be found in their respective github repositories.

Building

Clone the repo and type

$ ./mvnw clean install

For more information on building, see this link.

Contributing

We welcome contributions! Follow this link for more information on how to contribute.