Mirror of Apache Bahir
Scala Java Python Shell
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
bin [BAHIR-24] fix MQTT Python code, examples, add tests Jul 23, 2016
dev [BAHIR-162] Stop publishing MD5 hash with releases Apr 25, 2018
distribution [maven-release-plugin] prepare for next development iteration Jun 6, 2018
sql-cloudant [maven-release-plugin] prepare for next development iteration Jun 6, 2018
sql-streaming-akka [maven-release-plugin] prepare for next development iteration Jun 6, 2018
sql-streaming-mqtt [maven-release-plugin] prepare for next development iteration Jun 6, 2018
streaming-akka [maven-release-plugin] prepare for next development iteration Jun 6, 2018
streaming-mqtt [maven-release-plugin] prepare for next development iteration Jun 6, 2018
streaming-pubsub [maven-release-plugin] prepare for next development iteration Jun 6, 2018
streaming-twitter [maven-release-plugin] prepare for next development iteration Jun 6, 2018
streaming-zeromq [maven-release-plugin] prepare for next development iteration Jun 6, 2018
.gitattributes [MINOR] Re-add .gitignore and .gitattributes Mar 28, 2017
.gitignore [MINOR] Add checkpoint directory to git ignore configuration Jun 7, 2017
.travis.yml [BAHIR-163] Enable builds using Travis CI Apr 25, 2018
LICENSE Add project LICENSE and NOTICE files Jun 8, 2016
NOTICE [MINOR] update NOTICE file for 2017 Jan 6, 2017
README.md [BAHIR-101] Spark SQL datasource for CounchDB/Cloudant Mar 31, 2017
pom.xml [maven-release-plugin] prepare for next development iteration Jun 6, 2018
scalastyle-config.xml [MINOR] update ImportOrderChecker Jan 18, 2017

README.md

Apache Bahir

Apache Bahir provides extensions to distributed analytics platforms such as Apache Spark & Apache Flink.

http://bahir.apache.org/

Apache Bahir origins

The Initial Bahir source code (see issue BAHIR-1) containing the source for the Apache Spark streaming connectors for akka, mqtt, twitter, zeromq extracted from Apache Spark revision 8301fad (before the deletion of the streaming connectors akka, mqtt, twitter, zeromq).

Source code structure

Source code folder structure:

- streaming-akka
  - examples/src/main/...
  - src/main/...
- streaming-mqtt
  - examples
  - src
  - python
- ...

Building Bahir

Bahir is built using Apache Maven. To build Bahir and its example programs, run:

mvn -DskipTests clean install

Running tests

Testing first requires building Bahir. Once Bahir is built, tests can be run using:

mvn test

Example programs

Each extension currently available in Apache Bahir has an example application located under the "examples" folder.

Documentation

Currently, each submodule has its own README.md, with information on example usages and API.

Furthermore, to generate scaladocs for each module:

$ mvn package

Scaladocs is generated in, MODULE_NAME/target/site/scaladocs/index.html. __ Where MODULE_NAME is one of, sql-streaming-mqtt, streaming-akka, streaming-mqtt, streaming-zeromq, streaming-twitter. __

A note about Apache Spark integration

Currently, each module in Bahir is available through spark packages. Please follow linking sub section in module specific README.md for more details.