Mirror of Apache Gearpump (Incubating)
Scala JavaScript HTML Java Shell Python CSS
Latest commit 5498ae2 Feb 28, 2017 @manuzhang manuzhang [GEARPUMP-283] Return app exception to client
Author: manuzhang <owenzhang1990@gmail.com>

Closes #161 from manuzhang/GEARPUMP-283.
Permalink
Failed to load latest commit information.
.github [GEARPUMP-229] Add GitHub pull request template Dec 6, 2016
bin fixes GEARPUMP-159 Prep for first release Jun 30, 2016
conf [GEARPUMP-277] Allow user to configure retry times on app failure Feb 22, 2017
core/src [GEARPUMP-282] Allow user to configure wait duration for a running app Feb 24, 2017
dev-tools [GEARPUMP-243] Add missing license headers and DISCLAIMER Dec 3, 2016
docs [GEARPUMP-213] Fix EditOnGitHub link and rename docs/docs to docs/con… Jan 19, 2017
examples [GEARPUMP-103] Support finite stream Feb 10, 2017
experiments [GEARPUMP-236] Upgrade sbt-assembly version to fix java.lang.VerifyError Feb 23, 2017
external [GEARPUMP-268] Accept java.time.Instant in Message Feb 9, 2017
gearpump-hadoop/src/main [GEARPUMP-222] remove DFSJarStore from gearpump-daemon Oct 12, 2016
integrationtest [GEARPUMP-266] Add docker file Feb 8, 2017
licenses [GEARPUMP-240] Correct licenses Nov 16, 2016
notes Add 0.8.2 release notes and bump up version Dec 1, 2016
project [GEARPUMP-236] Upgrade sbt-assembly version to fix java.lang.VerifyError Feb 23, 2017
services [GEARPUMP-270] Disable 'Kill' button when applciation is not running Feb 13, 2017
streaming/src [GEARPUMP-283] Return app exception to client Feb 28, 2017
unmanagedlibs GEARPUMP-11, fix code style Apr 26, 2016
yarnconf GEARPUMP-11, fix code style Apr 26, 2016
.coveragerc fix #132 add codecov config Dec 25, 2014
.gitignore [GEARPUMP-213] build docs with MkDocs Sep 29, 2016
.rat-excludes [GEARPUMP-183] fix licenses Jul 30, 2016
.sbtopts GEARPUMP-9, Clean and fix integration test Apr 26, 2016
.travis.yml [GEARPUMP-273] Remove java in travis language setting Feb 16, 2017
CHANGELOG.md Add 0.8.2 release notes and bump up version Dec 1, 2016
CONTRIBUTING.md GEARPUMP-244 Fixed the broken link of IDE setup guide Dec 5, 2016
DISCLAIMER Add disclaimer Jul 15, 2016
KEYS fixes GEARPUMP-159 Prep for first release Jun 30, 2016
LICENSE [GEARPUMP-227] remove unneeded licenses Oct 29, 2016
LICENSE.bin [GEARPUMP-240] Correct licenses Nov 16, 2016
NOTICE [GEARPUMP-243] Add missing license headers and DISCLAIMER Dec 3, 2016
README.md [DOC] Add Karol Brejna to contributors Feb 16, 2017
ReleaseProcess.md Fixes GEARPUMP-234 Update LICENSE and related files for 0.8.2-incubat… Nov 12, 2016
codecov.yml fix GEARPUMP-108 enable travis CI for Gearpump's apache repository May 5, 2016
jvm.sbt GEARPUMP-11, fix code style Apr 26, 2016
pubring.asc fix #359, Allow travis to auto publish signed artifacts to nexus sona… Jan 27, 2015
scalastyle-config.xml GEARPUMP-11, fix code style Apr 26, 2016
secring.asc.enc fix #359, Allow travis to auto publish signed artifacts to nexus sona… Jan 27, 2015
version.sbt Add 0.8.2 release notes and bump up version Dec 1, 2016

README.md

Apache Gearpump (incubating) Maven Central GitHub license Build Status codecov.io

Gearpump is a lightweight real-time big data streaming engine. It is inspired by recent advances in the Akka framework and a desire to improve on existing streaming frameworks.

The name Gearpump is a reference to the engineering term "gear pump", which is a super simple pump that consists of only two gears, but is very powerful at streaming water.

We model streaming within the Akka actor hierarchy.

Per initial benchmarks we are able to process near 18 million messages/second (100 bytes per message) with a 8ms latency on a 4-node cluster.

For steps to reproduce the performance test, please check Performance benchmark.

Useful Resources

How to Build

1). Clone the Gearpump repository

  git clone https://github.com/apache/incubator-gearpump.git
  cd gearpump

2). Build package

  ## Please use scala 2.11 or 2.10
  ## The target package path: output/target/gearpump-${version}.zip
  sbt clean +assembly +packArchiveZip

After the build, there will be a package file gearpump-${version}.zip generated under output/target/ folder.

To build scala document, use

   ## Will generate the scala doc under target/scala_2.xx/unidoc/
   sbt unidoc

NOTE: The build requires network connection. If you are behind an enterprise proxy, make sure you have set the proxy in your env before running the build commands. For windows:

set HTTP_PROXY=http://host:port
set HTTPS_PROXY= http://host:port

For Linux:

export HTTP_PROXY=http://host:port
export HTTPS_PROXY= http://host:port

How to run Gearpump integration test

Gearpump has an integration test system which is based on Docker. Please check the instructions.

How to do style check before submitting a pull request?

Before submitting a PR, you should always run style check first:

  ## Run style check for compile, test, and integration test.
  sbt scalastyle test:scalastyle it:scalastyle

How to generate the license report to generate a list of all dependencies

  sbt dumpLicenseReport

How to generate dependencies by declared license

  sbt dependencyLicenseInfo

Contributors (time order)

Contacts

License

Gearpump itself is licensed under the Apache License (2.0). For library it used, please see LICENSE.

Acknowledgement

The netty transport code work is based on Apache Storm. Thanks Apache Storm contributors.

The cgroup code work is based on JStorm. Thanks JStorm contributors.

Thanks to Jetbrains for providing a IntelliJ IDEA Free Open Source License.