Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
A file busing system for integration with Hadoop
Java
Branch: master
#2 Compare This branch is 7 commits ahead of esammer:master.

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
.settings
src
.classpath
.gitignore
.project
.springBeans
README.md
pom.xml

README.md

fbus - A file busing system

Overview

FBus is a file bussing system based on Spring and Spring Integration. This project is extremely experimental and is meant to:

  • Demonstrate one way to integration Hadoop / HDFS with existing systems using file system queues.
  • Scratch an itch.

The hope is that people find this useful as both an example and a project. If this is true, development will continue.

Building

This is a Maven project and can be built by running mvn assembly:assembly. A number of artifacts will be available in the target directory including a jar without the dependencies and one with the dependencies included.

The Spring Source bundle repository is used for Spring dependencies. The (currently unpublished) Cloudera repository is used for Hadoop and its dependencies. Specifically, this builds against CDH3u3. Some dependencies are pulled from Maven Central, but this will move to the Spring Source repository to reduce potential metadata issues.

Configuration

See the example configuration fbus.xml in src/test/resources for a sample of how to wire together various components. Additional docs forthcoming.

Author

E. Sammer - esammer@cloudera.com

Copyright / License / Disclaimer

This code is released under the Apache Software License. While some package names contain the name "Cloudera," this software is unsupported by Cloudera. I just happened to work there and needed a unique package space to use during development.

Absolutely everything is subject to change and comes without warranty.

Something went wrong with that request. Please try again.