A logdriver for Apache Hadoop
Java Perl Shell
Switch branches/tags
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
bin
conf.example
doc
oozie/workflows
pig
src
testcases/logsearch
.gitignore
LICENSE
README.md
boom.schema.json
logindex.schema.json
pom.xml

README.md

Overview

LogDriver suite of tools and applications for logging management on Hadoop. It is primary built and maintained at BlackBerry, but we welcome contributions in any form from the larger community.

Major Features

  • Sawmill: A low profile, high throughput syslog adapter designed as a drop-in replacement for Flume for delivery of syslog content to HDFS.
  • Merge Job: An Oozie workflow which takes content delivered by Sawmill and performs a lightweight aggregation of the content into large files.
  • Filter Job: An Oozie workflow which will apply data lifecycle rules, filtering and deleting data according to configuration.
  • Data Access Tools: A set of tools designed to be executed from the command line, replacing common log management functions such as cat and grep.
  • Data Reporting Tools: A set of tools designed to collect statistical metadata of various log content.
  • Strong Security Model: Full support for Kerberos+LDAP based authentication and strict security models.

For further information, please see our Documentation