Skip to content
Go stream processing library
Branch: master
Clone or download
Latest commit 2a92bbc Jun 6, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
examples add redis example May 12, 2019
extension use log package for logging May 13, 2019
flow fix bugs and improve coverage Jun 5, 2019
images readme: add pipeline architecture example May 13, 2019
.gitignore Initial commit Apr 30, 2019
.travis.yml remove race flag Jun 5, 2019
LICENSE Initial commit Apr 30, 2019
README.md integrate codecov Jun 5, 2019
go.mod use go mod Jun 4, 2019
go.sum use go mod Jun 4, 2019
streams.go Initial commit Apr 30, 2019
util.go add HashCode function May 10, 2019

README.md

go-streams

Build Status GoDoc Go Report Card codecov

Go stream processing library.
Provides simple and concise DSL to build data pipelines. pipeline-architecture-example

Wiki
In computing, a pipeline, also known as a data pipeline,[1] is a set of data processing elements connected in series, where the output of one element is the input of the next one. The elements of a pipeline are often executed in parallel or in time-sliced fashion. Some amount of buffer storage is often inserted between elements.

Overview

Building blocks:

  • Source - A Source is a set of stream processing steps that has one open output.
  • Flow - A Flow is a set of stream processing steps that has one open input and one open output.
  • Sink - A Sink is a set of stream processing steps that has one open input. Can be used as a Subscriber.

Flow capabilities (flow package):

  • Map
  • FlatMap
  • PassThrough
  • Split
  • FanOut
  • Merge
  • Throttler
  • SlidingWindow
  • TumblingWindow

Supported Sources and Sinks (ext package):

Examples

Could be found in the examples directory.

License

Licensed under the MIT License.

You can’t perform that action at this time.