Skip to content
master
Switch branches/tags
Code

Latest commit

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ws
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

go-streams

Build Status PkgGoDev Go Report Card codecov

A lightweight stream processing library for Go.
go-streams provides a simple and concise DSL to build data pipelines. pipeline-architecture-example

Wiki
In computing, a pipeline, also known as a data pipeline, is a set of data processing elements connected in series, where the output of one element is the input of the next one. The elements of a pipeline are often executed in parallel or in time-sliced fashion. Some amount of buffer storage is often inserted between elements.

Overview

Building blocks:

  • Source - A Source is a set of stream processing steps that has one open output.
  • Flow - A Flow is a set of stream processing steps that has one open input and one open output.
  • Sink - A Sink is a set of stream processing steps that has one open input. Can be used as a Subscriber.

Flow capabilities (flow package):

  • Map
  • FlatMap
  • Filter
  • PassThrough
  • Split
  • FanOut
  • Merge
  • Throttler
  • SlidingWindow
  • TumblingWindow

Supported Connectors:

Examples

Usage samples are available in the examples directory.

License

Licensed under the MIT License.