Skip to content

streamr-dev/streamr-spark-integrations

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Apache spark integrations from Streamr

You can find the maven library for the integration here You can import it to your maven project easily with:

<dependency>
  <groupId>com.streamr.labs</groupId>
  <artifactId>streamr_spark</artifactId>
  <version>0.2</version>
</dependency>

This repository contains different ways to integrate Streamr to Apache Spark.

Direct integrations from Streamr to Spark are done with Streamr's Java client for Java and Scala versions of Spark. Non-direct example integrations for Streamr-NiFi-Kafka-Spark-Kafka-NiFi-Streamr and a Streamr integration to Spark file streaming are also provided. The NiFi + Kafka integration might be useful for people who wish to either subscribe or publish data via NiFi and/or Kafka. The file streaming JS integration tool is useful for people who wish to do operations on historical data from Streamr and possibly store it for later use.

The direct integrations use the Helsinki Tram Stream as their data input.

Java Direct Integration

Scala Integrations

JS file integration tool

About

Example integrations from Streamr to Apache Spark

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published