Apache StreamPipes (incubating) enables flexible modeling of stream processing pipelines by providing a graphical modeling editor on top of existing stream processing frameworks.
It leverages non-technical users to quickly define and execute processing pipelines based on an easily extensible toolbox of data sources, data processors and data sinks. StreamPipes has an exchangeable runtime execution layer and executes pipelines using one of the provided wrappers, e.g., for Apache Flink or Apache Kafka Streams.
Pipeline elements in StreamPipes can be installed at runtime - the built-in SDK allows to easily implement new pipeline elements according to your needs. Pipeline elements are standalone microservices that can run anywhere - centrally on your server, in a large-scale cluster or close at the edge.
Learn more about StreamPipes at https://streampipes.apache.org/
Read the full documentation at https://streampipes.apache.org/docs
StreamPipes Pipeline Elements
This project provides a library of several pipeline elements that can be used within the Apache StreamPipes toolbox.
See https://streampipes.apache.org/docs/docs/pipeline-elements/ for an overview of currently available pipeline elements.
Contact us if you are missing some pipeline elements!
See the installation guide at https://streampipes.apache.org/download.html
It's easy to get started:
- Download the installer script from https://github.com/apache/incubator-streampipes-installer
- Follow the installation guide at https://streampipes.apache.org/docs/docs/user-guide-installation
- Check the tour and build your first pipeline!
If you have any problems during the installation or questions around StreamPipes, you'll get help through one of our community channels:
And don't forget to follow us on Twitter!
We welcome contributions to StreamPipes. If you are interested in contributing to StreamPipes, let us know (e.g ., send an email to the dev list)!