Build and scale real-time data applications as easily as writing a Python script
Clone or download
Permalink
Failed to load latest commit information.
.ci-dockerfiles/ci-standard Refactor testing/tools/integration (#2375) Sep 5, 2018
.circleci Update CircleCI Docker image tag Aug 13, 2018
.for_maintainers Update Maintainer Release Docs (#2372) Aug 28, 2018
.github Add Issue and PR github templates (#2373) Aug 30, 2018
.release Update docker_release.sh to tag/push latest images on release branch … Sep 20, 2018
book Add a book chapter on connectors (#2412) Sep 20, 2018
connectors Fix a few connector issues and add an example app using UDP (#2408) Sep 19, 2018
demos Fix Go API json naming problem Sep 19, 2018
docker More release related updates (#2357) Aug 20, 2018
examples Add a book chapter on connectors (#2412) Sep 20, 2018
giles Add muti_partition_detector and integration test Jul 17, 2018
go_api Use static state step actor graph for dynamic keys Sep 19, 2018
lib Add missing .cstring() Sep 20, 2018
machida Fix a few connector issues and add an example app using UDP (#2408) Sep 19, 2018
misc Update wallaroo up for python changes and add ci example testing. Sep 19, 2018
monitoring_hub Remove old/unnecessary static files from metrics ui. Aug 20, 2018
orchestration Fix Ansible AWS configure issue Mar 22, 2018
testing Updates to perf testing docs Sep 20, 2018
travis Update wallaroo up for python changes and add ci example testing. Sep 19, 2018
utils Fix bitrot in utils/resilience-demo/6* Sep 19, 2018
vagrant Update version for 0.5.2 release Aug 24, 2018
.gitignore Fixes to test logic Sep 19, 2018
.travis.yml Update wallaroo up for python changes and add ci example testing. Sep 19, 2018
CHANGELOG.md Add unreleased section to CHANGELOG Aug 24, 2018
CODE_OF_CONDUCT.md Fix formatting in COC Sep 7, 2017
CONTRIBUTING.md Fix typo. Aug 10, 2018
Dockerfile Fix release scripts and add Dockerfile. Aug 24, 2018
LICENSE.md Rename LICENSE to LICENSE.md Sep 14, 2017
LIMITATIONS.md Add to_sinks support to Python API Feb 22, 2018
MONOREPO.md Update readme (#2327) Sep 5, 2018
Makefile Have `make test` run `make unit-tests` and `make integration-tests` Oct 2, 2017
README.md Update readme (#2327) Sep 5, 2018
ROADMAP.md Update ROADMAP based on changes in plans Jan 26, 2018
SUMMARY.md Add resilience.md section in running-wallaro Sep 20, 2018
SUPPORT.md Fix incorrect IRC link Nov 1, 2017
VERSION Update version for 0.5.2 release Aug 24, 2018
book.json Step 1 towards automated changing of version number in book. Jan 9, 2018
cover.jpg Another cover.jpg size Sep 19, 2017
intro.md Fix typo Jun 11, 2018
rules.mk Add resilience tests Sep 19, 2018
wallaroo-logo.png Add Wallaroo logo to README Dec 20, 2017

README.md

WallarooLabs logo

Build and scale real-time applications as easily as writing a script


CircleCI GitHub license GitHub version IRC Groups.io

A fast, stream-processing framework. Wallaroo makes it easy to react to data in real-time. By eliminating infrastructure complexity, going from prototype to production has never been simpler.

Answer our 30-second survey, and be entered to win a Wallaroo T-Shirt.

What is Wallaroo?

When we set out to build Wallaroo, we had several high-level goals in mind:

  • Create a dependable and resilient distributed computing framework
  • Take care of the complexities of distributed computing "plumbing," allowing developers to focus on their business logic
  • Provide high-performance & low-latency data processing
  • Be portable and deploy easily (i.e., run on-prem or any cloud)
  • Manage in-memory state for the application
  • Allow applications to scale as needed, even when they are live and up-and-running

You can learn more about Wallaroo from our "Hello Wallaroo!" blog post and the Wallaroo overview video.

What makes Wallaroo unique

Wallaroo is a little different than most stream processing tools. While most require the JVM, Wallaroo can be deployed as a separate binary. This means no more jar files. Wallaroo also isn't locked to just using Kafka as a source, use any source you like. Application logic can be written in either Python 2, Go, or Pony; with more coming soon.

Getting Started

Wallaroo can either be installed via Docker, Vagrant or (on Linux) compiled from source.

As easy as:

docker pull wallaroo-labs-docker-wallaroolabs.bintray.io/release/wallaroo:0.5.2

Usage

Once you've installed Wallaroo, Take a look at some of our examples. A great place to start are our reverse or market spread examples in either Python or Go.

"""
This is an example application that receives strings as input and outputs the
reversed strings.
"""

def application_setup(args):
  # see ./examples/python/reverse/ for the full example and how to run it

@wallaroo.decoder(header_length=4, length_fmt=">I")
def decoder(bs):
    return bs.decode("utf-8")

@wallaroo.computation(name="reverse")
def reverse(data):
    return data[::-1]

@wallaroo.encoder
def encoder(data):
    # data is a string
    return data + "\n"

Documentation

Are you the sort who just wants to get going? Dive right into our documentation then! It will get you up and running with Wallaroo.

More information is also on our blog. There you can find more insight into what we are working on and industry use-cases.

Wallaroo currently exists as a mono-repo. All the source that is Wallaroo is located in this repo. See application structure for more information.

Need Help?

Trying to figure out how to get started?

Contributing

We welcome contributions. Please see our Contribution Guide

For your pull request to be accepted you will need to accept our Contributor License Agreement

License

The Wallaroo Community License is based on Apache version 2. However, you should read it for yourself. Here we provide a summary of the main points of the Wallaroo Community License Agreement.

  • You can run all Wallaroo code in a non-production environment without restriction.
  • You can run all Wallaroo code in a production environment for free on up to 3 server or 24 cpus.
  • If you want to run Wallaroo Enterprise version features in production above 3 servers or 24 cpus, you have to obtain a license.
  • You can modify and redistribute any Wallaroo code
  • Anyone who uses your modified or redistributed code is bound by the same license and needs to obtain a Wallaroo Enterprise license to run on more than 3 servers or 24 cpus in a production environment.