A lightweight stream processing library for Go.
go-streams provides a simple and concise DSL to build data pipelines.
In computing, a pipeline, also known as a data pipeline, is a set of data processing elements connected in series, where the output of one element is the input of the next one. The elements of a pipeline are often executed in parallel or in time-sliced fashion. Some amount of buffer storage is often inserted between elements.
- Source - A Source is a set of stream processing steps that has one open output.
- Flow - A Flow is a set of stream processing steps that has one open input and one open output.
- Sink - A Sink is a set of stream processing steps that has one open input. Can be used as a Subscriber.
Flow capabilities (flow package):
- Go channels
- File system
- Network (TCP, UDP)
- Apache Kafka
- Apache Pulsar
Usage samples are available in the examples directory.
Licensed under the MIT License.