Java XSLT Other
Latest commit 16edf43 Aug 13, 2018
Permalink
Failed to load latest commit information.
.mvn Update to maven 3.5 Apr 20, 2017
spring-cloud-dataflow-classic-docs Implement concurrent task execution limit Aug 1, 2018
spring-cloud-dataflow-completion Change project version to 1.7.0.BUILD-SNAPSHOT Jul 31, 2018
spring-cloud-dataflow-configuration-metadata Change project version to 1.7.0.BUILD-SNAPSHOT Jul 31, 2018
spring-cloud-dataflow-core Change project version to 1.7.0.BUILD-SNAPSHOT Jul 31, 2018
spring-cloud-dataflow-dependencies Change project version to 1.7.0.BUILD-SNAPSHOT Jul 31, 2018
spring-cloud-dataflow-docs Use 1.3 apps, expose port range of 9000-9010 Aug 13, 2018
spring-cloud-dataflow-registry Change project version to 1.7.0.BUILD-SNAPSHOT Jul 31, 2018
spring-cloud-dataflow-rest-client Add 'spring-boot-configuration-processor' dependency Aug 13, 2018
spring-cloud-dataflow-rest-resource Adds Restful Client for Scheduler calls Aug 1, 2018
spring-cloud-dataflow-server-core Move stream aggregate state logic into Util Aug 14, 2018
spring-cloud-dataflow-server-local-autoconfig Change project version to 1.7.0.BUILD-SNAPSHOT Jul 31, 2018
spring-cloud-dataflow-server-local Use 1.3 apps, expose port range of 9000-9010 Aug 13, 2018
spring-cloud-dataflow-shell-core Re-add testCurrentRunningTasks test to TestCommands Aug 13, 2018
spring-cloud-dataflow-shell Change project version to 1.7.0.BUILD-SNAPSHOT Jul 31, 2018
spring-cloud-dataflow-skipper-docs Change project version to 1.7.0.BUILD-SNAPSHOT Jul 31, 2018
spring-cloud-starter-dataflow-server-local Implement concurrent task execution limit Aug 1, 2018
src Added AnnotationUseStyle to checkstyle config Mar 6, 2018
.gitignore Implement concurrent task execution limit Aug 1, 2018
.settings.xml Restore tabs for indentation in XML files Apr 28, 2017
.travis.yml Stream updates should reflect the state in SCDF correctly Dec 29, 2017
CODE_OF_CONDUCT.adoc Add Code of Conduct Apr 27, 2016
LICENSE maven infrastructure Jul 21, 2015
README.adoc Add reporting-lissues wiki link Nov 17, 2017
mvnw GH-163, GH-173, GH-215 Add travis, mvnw, and update README Dec 8, 2015
mvnw.cmd #642 Maven Wrapper cmd script issue fixed Jun 3, 2016
pom.xml Implement concurrent task execution limit Aug 1, 2018

README.adoc

Latest Release Version Latest Snapshot Version Codecov

Spring Cloud Data Flow Build Status Stories Ready Stories In Progress

Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines.

Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks.

This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics.

Components

The Core domain module includes the concept of a stream that is a composition of spring-cloud-stream modules in a linear pipeline from a source to a sink, optionally including processor application(s) in between. The domain also includes the concept of a task, which may be any process that does not run indefinitely, including Spring Batch jobs.

The App Registry maintains the set of available apps, and their mappings to URIs. For example, if relying on Maven coordinates, an app’s URI would be of the format: maven://<groupId>:<artifactId>:<version>

The Data Flow Server is a Spring Boot application that provides a common REST API and UI. For each runtime environment there is a different version of the Data Flow Server that depends upon a deployer SPI implementation for that environment. The github locations for these Data Flow Servers are:

There are also community maintained Spring Cloud Data Flow implementations for Hashicorp’s Nomad and RedHat’s Openshift.

The deployer SPI mentioned above is defined within the Spring Cloud Deployer project. That provides an abstraction layer for deploying the apps of a given stream or task and managing their lifecycle. The github locations for the corresponding Spring Cloud Deployer SPI implementations are:

The Shell connects to the Data Flow Server’s REST API and supports a DSL that simplifies the process of defining a stream or task and managing its lifecycle.

Instructions for running the Data Flow Server for each runtime environment can be found in their respective github repositories.

Building

Clone the repo and type

$ ./mvnw clean install

For more information on building, see this link.

Building on Windows

When using Git on Windows to check out the project, it is important to handle line-endings correctly during checkouts. By default Git will change the line-endings during checkout to CRLF. This is, however, not desired for Spring Cloud Data Flow as this may lead to test failures under Windows.

Therefore, please ensure that you set Git property core.autocrlf to false, e.g. using: $ git config core.autocrlf false. Fore more information please refer to the Git documentation, Formatting and Whitespace.

Contributing

We welcome contributions! Follow this link for more information on how to contribute.

Reporting Issues

When reporting problems, it’d be helpful if the bug report includes the details listed on this wiki-article.

Code formatting guidelines

  • The directory ./src/eclipse has two files for use with code formatting, eclipse-code-formatter.xml for the majority of the code formatting rules and eclipse.importorder to order the import statements.

  • In eclipse you import these files by navigating Windows → Preferences and then the menu items Preferences > Java > Code Style > Formatter and Preferences > Java > Code Style > Organize Imports respectfully.

  • In IntelliJ, install the plugin Eclipse Code Formatter. You can find it by searching the "Browse Repositories" under the plugin option within IntelliJ (Once installed you will need to reboot Intellij for it to take effect). Then navigate to Intellij IDEA > Preferences and select the Eclipse Code Formatter. Select the eclipse-code-formatter.xml file for the field Eclipse Java Formatter config file and the file eclipse.importorder for the field Import order. Enable the Eclipse code formatter by clicking Use the Eclipse code formatter then click the OK button.

    • NOTE: If you configure the Eclipse Code Formatter from File > Other Settings > Default Settings it will set this policy across all of your Intellij projects.