Skip to content
Cache File System optimized for columnar formats and object stores
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
assembly
docs chg: doc: #262: Update data engine documentation (#263) Mar 8, 2019
rubix-bookkeeper
rubix-client
rubix-common fix: dev: #267: Added license file and updated copyright header (#268) Mar 11, 2019
rubix-core
rubix-hadoop2 fix: dev: #269: Fix security related issues (#266) Mar 14, 2019
rubix-presto
rubix-rpm
rubix-spi fix: dev: #269: Fix security related issues (#266) Mar 14, 2019
rubix-tests
src/checkstyle fix: dev: #267: Added license file and updated copyright header (#268) Mar 11, 2019
.gitchangelog.rc
.gitignore
.travis.yml new : dev: #165: Add unit test to run daemons in separate jvm in a do… Aug 7, 2018
CHANGELOG Add gitchangelog rc and CHANGELOG (#150) May 27, 2018
CONTRIBUTING.md new:usr:Add more details to Contribution Guide and a markdown file (#145 May 23, 2018
Dockerfile
LICENSE.txt
README.md chg: doc: #262: Update data engine documentation (#263) Mar 8, 2019
codecov.yaml
pom.xml fix: dev: #267: Added license file and updated copyright header (#268) Mar 11, 2019
script.sh new: dev: #250: Add support to check bookkeeper connectivity (#251) Jan 6, 2019

README.md

RubiX

Build Status codecov

RubiX is a light-weight data caching framework that can be used by Big-Data engines. RubiX uses local disks to provide the best I/O bandwidth to the Big Data Engines. RubiX is useful in shared storage architectures where the data execution engine is separate from storage. For example, on public clouds like AWS or Microsoft Azure, data is stored in cloud store and the engine accesses the data over a network. Similarly in data centers Presto runs on a separate cluster from HDFS and accesses data over the network.

RubiX can be extended to support any engine that accesses data using Hadoop FileSystem interface via plugins. There are plugins to access data on AWS S3, Microsoft Azure Blob Store and HDFS. RubiX can be extended to be used with any other storage systems including other cloud stores

Check the User and Developer manual for more more information on getting started.

Supported Engines and Cloud Stores

  • Presto: Amazon S3
  • Spark: Amazon S3
  • Any engine using hadoop-2 e.g. Hive can utilize RubiX. Amazon S3 is supported

Resources

Documentation
Getting Started Guide
User Group (Google)

Talks

Talk on Rubix at Strata 2017

Blog Posts

Developers

Slack Channel

The channel is restricted to a few domains. Send an email on the user group or contact us through Github issues. We will add you to the slack channel.

You can’t perform that action at this time.