Skip to content

0.6.1

Latest
Latest
Compare
Choose a tag to compare
@JONBRWN JONBRWN released this 31 Dec 16:00
· 474 commits to master since this release
0.6.1
23babad

Release Notes

We’re excited to announce the addition of stream windowing to the Wallaroo API. You can now aggregate data in count or range-based windows. For example, the following code sample defines 6-second sliding windows starting every 3 seconds for a user-defined aggregation:

    (inputs
        .to(wallaroo.range_windows(wallaroo.seconds(6))
            .with_slide(wallaroo.seconds(3))
            .over(MyAgg))
        .to_sink(sink_config))

This means first-class support for the wide range of streaming use cases that require windowing.

We've also updated our documentation site! If you happen to run into any problems when navigating the new site, feel free to open an issue!

What is Wallaroo

Wallaroo is a modern, extensible framework that makes it simple to get stateful streaming data and event-driven applications to production fast, regardless of scale.

If you are interested in installing Wallaroo, our installation documentation provides the various ways you can get up and running.

Feel free to use the table of contents below to help you navigate to sections you might find relevant.

Table of Contents

  1. New Features and Improvements
  2. Installing Wallaroo
  3. Upgrading Wallaroo
  4. CHANGELOG

New Features and Improvements

Aggregations

Aggregations are an alternative to state computations that trade some of the freedom provided by state computations for the ability to efficiently compute results in windows. This goes hand-in-hand with our new Windowing API, which is also part of this release.

Windowing API

Wallaroo now supports windowing over aggregations. This allows you to break an infinite stream into manageable chunks and also allows you to see how your inputs evolve over time.
There are two broad categories of windows currently supported: count-based and range-based. Count-based windows emit an output every n input messages, where n is specified via the API. Range-based windows are based on fixed time ranges, and can be further divided into tumbling and sliding (overlapping) varieties. See our documentation for an in-depth description of these concepts and our API.

API Update: collect()

We pass an implicit routing key along each stage in a Wallaroo pipeline. This release adds a new API call ".collect()" for the case where you want all messages after a certain stage to be assigned the same routing key. Prior to this change, you had to use a key_by call that used a constant key_extractor function. This was onerous and also limited our options to optimize topology layouts under the hood.

Installing Wallaroo

There are currently four ways that you can install Wallaroo:
The Wallaroo Up script
Docker
Vagrant

Upgrading Wallaroo

In all cases below, if you run into issues, please reach out to us! We’re available on twitter, IRC, Github, by email, our mailing list, or our subreddit. We love questions!

If you have made no changes to Wallaroo or Pony since installation, your best bet will be to start from scratch, following the instructions of your choice.

Below are instructions for Upgrading Wallaroo via Wallaroo Up, Upgrading Wallaroo in Docker, and Upgrading Wallaroo in Vagrant.

Upgrading Wallaroo via Wallaroo Up

The normal Wallaroo Up installation instructions will install new versions next to existing versions.

Upgrading the Wallaroo Docker image

To upgrade the Wallaroo Docker image, run the following command to get the latest image. If you don't allow a non-root user to run Docker commands, you'll need to add sudo to the front of the command.

docker pull wallaroo-labs-docker-wallaroolabs.bintray.io/release/wallaroo:0.6.1

Upgrading Wallaroo Source Code

If you mounted the Wallaroo source code to your local machine using the directory recommended in setup, in /tmp/wallaroo-docker (UNIX & MacOS users) or c:/wallaroo-docker (Windows users), then you will need to move the existing directory in order to get the latest source code. The latest Wallaroo source code will be copied to this directory automatically when a new container is started with the latest Docker image.

UNIX & MacOS Users

For UNIX users, you can move the directory with the following command:

mv /tmp/wallaroo-docker/wallaroo-src/ /tmp/wallaroo-docker/wallaroo-0.6.0-src/
Windows Users

For Windows users, you can move the directory with the following command:

move c:/wallaroo-docker/wallaroo-src/ c:/wallaroo-docker/wallaroo-0.6.0-src

Once done moving, you can re-create the wallaroo-src directory with the following command:

mkdir c:\wallaroo-docker\wallaroo-src

Upgrading Wallaroo in Vagrant

To upgrade your Wallaroo installation in Vagrant, you’ll want to follow the latest installation instructions for Wallaroo in Vagrant.

If you have modified your old Vagrant VM in any way that you intend to persist, you should persist your changes now. For example, copy any edited or new files from the old Vagrant VM to the new one.

CHANGELOG

[0.6.1] - 2018-12-31

Added

Assets 2