Onyx

What is it?

• a masterless, cloud scale, fault tolerant, high performance distributed computation system
• batch and stream hybrid processing model
• exposes an information model for the description and construction of distributed workflows
• Competes against Storm, Cascading, Cascalog, Spark, Map/Reduce, Sqoop, etc
• written in pure Clojure

What would I use this for?

• Realtime event stream processing
• Continuous computation
• Extract, transform, load
• Data transformation à la map-reduce
• Data ingestion and storage medium transfer
• Data cleaning

Installation

Available on Clojars:

[org.onyxplatform/onyx "0.7.3"]

Build Status

Component	0.7.x	master
onyx core
onyx-sql
onyx-datomic
onyx-kafka
onyx-durable-queue
onyx-seq

Quick Start Guide

Feeling impatient? Hit the ground running ASAP with the onyx-starter repo and walkthrough. You can also boot into preloaded a Leiningen application template.

User Guide 0.7.3

• User Guide HTML
• User Guide PDF
• User Guide Website

API Docs 0.7.3

Code level API documentation can be found here.

Official plugin listing

Official plugins are vetted by Michael Drogalis. Ensure in your project that plugin versions directly correspond to the same Onyx version (e.g. onyx-core-async version 0.7.3 goes with onyx version 0.7.3). Fixes to plugins can be applied using a 4th versioning identifier (e.g. 0.7.3.1).

• onyx-core-async
• onyx-datomic
• onyx-sql
• onyx-kafka
• onyx-s3
• onyx-durable-queue

Generate plugin templates through Leiningen with onyx-plugin.

Offical Dashboard and Metrics

You can run a dashboard to monitor Onyx cluster activity, found here. Further, you can collect metrics and send them to the dashboard, or anywhere, by using the onyx-metrics plugin.

Release Notes

You can find the latest major release notes here.

Need help?

Check out the Onyx Google Group.

Want the logo?

Feel free to use it anywhere. You can find a few different versions here.

Running the tests

A simple lein midje will run the full suite.

Contributing

Contributions are welcome. Please fork the repository and send a pull request to the master branch.

Branching

Onyx uses a similiar branching strategy to Clojure itself. Onyx uses semantic versioning, and each minor version gets its own branch. All work is done on develop or feature branches and dropped into a major.minor.x branch when it's time to cut a new release. Pull requests into the develop branch are welcome.

Commit rights

Anyone who has a patch accepted may request commit rights. Please do so inside the pull request post-merge.

Contributor list

• Michael Drogalis
• Owen Jones
• Bruce Durling
• Malcolm Sparks
• Lucas Bradstreet
• Bryce Blanton
• David Rupp
• sbennett33
• Tyler van Hensbergen
• David Leatherman
• Daniel Compton
• Jeff Rose
• Ole Krüger

Acknowledgements

Some code has been incorporated from the following projects:

• [Riemann] (https://github.com/aphyr/riemann)
• [zookeeper-clj] (https://github.com/liebke/zookeeper-clj)

Author

The lead for this project is Michael Drogalis, the original creator. You can get me directly at (mjd3089.at.rit.edu) if needed.

License

Copyright © 2015 Michael Drogalis

Distributed under the Eclipse Public License, the same as Clojure.

Profiler

YourKit supports open source projects with its full-featured Java Profiler. YourKit, LLC is the creator of YourKit Java Profiler and YourKit .NET Profiler, innovative and intelligent tools for profiling Java and .NET applications.