Mirror of Apache Flume
Java Shell PowerShell Python Thrift Protocol Buffer
Latest commit 4b6b841 Nov 18, 2016 @simonati simonati committed with bessbd Add Impala and Hive recommendation on hdfs.writeFormat setting in the…
… user guide

This closes #89

Reviewers: Denes Arvay, Jeff Holoman

(Attila Simon via Bessenyei Balázs Donát)
Permalink
Failed to load latest commit information.
bin FLUME-2913. Don't strip SLF4J from imported classpaths May 27, 2016
conf FLUME-2954. Make raw data appearing in log messages explicit Aug 29, 2016
dev-docs Change how we build and sign release artifacts Jan 29, 2017
dev-support Change how we build and sign release artifacts Jan 29, 2017
flume-checkstyle FLUME-2924. Update POM versions to 1.8.0-SNAPSHOT Oct 10, 2016
flume-ng-auth FLUME-3049. Make HDFS sink rotate more reliably in secure mode Feb 1, 2017
flume-ng-channels FLUME-3027. Change Kafka Channel to clear offsets map after commit Dec 6, 2016
flume-ng-clients FLUME-2924. Update POM versions to 1.8.0-SNAPSHOT Oct 10, 2016
flume-ng-configuration Add an HTTP sink Jan 24, 2017
flume-ng-core FLUME-3031. Change sequence source to reset its counter for event bod… Dec 6, 2016
flume-ng-dist Add an HTTP sink Jan 24, 2017
flume-ng-doc Add Impala and Hive recommendation on hdfs.writeFormat setting in the… Feb 17, 2017
flume-ng-embedded-agent FLUME-2924. Update POM versions to 1.8.0-SNAPSHOT Oct 10, 2016
flume-ng-legacy-sources FLUME-2924. Update POM versions to 1.8.0-SNAPSHOT Oct 10, 2016
flume-ng-node FLUME-2993. Add support for environment variables in configuration files Jan 6, 2017
flume-ng-sdk FLUME-2924. Update POM versions to 1.8.0-SNAPSHOT Oct 10, 2016
flume-ng-sinks FLUME-2427. Be less verbose in isFileClosed() notification Feb 17, 2017
flume-ng-sources FLUME-2857. Make Kafka Source/Channel/Sink restore default values whe… Oct 28, 2016
flume-ng-tests FLUME-2924. Update POM versions to 1.8.0-SNAPSHOT Oct 10, 2016
flume-shared FLUME-2924. Update POM versions to 1.8.0-SNAPSHOT Oct 10, 2016
flume-tools FLUME-2924. Update POM versions to 1.8.0-SNAPSHOT Oct 10, 2016
.gitignore FLUME-2206. ElasticSearchSink ttl field modification to mimic Elastic… Oct 31, 2013
.travis.yml Set MAVEN_OPTS for Travis-CI Oct 24, 2016
CHANGELOG Update CHANGELOG for 1.7 Oct 10, 2016
CONTRIBUTING.md Add Developer Section / How to Contribute from cwiki.apache.org Oct 27, 2016
DEVNOTES Add MAVEN_OPTS build workaround to DEVNOTES for 1.2.0 Jul 11, 2012
LICENSE Update LICENSE Oct 12, 2016
NOTICE Updated copyright year in NOTICE file. Jul 8, 2012
README.md Fix broken link in README Jul 19, 2016
RELEASE-NOTES Update RELEASE-NOTES for 1.7.0 Oct 10, 2016
doap_Flume.rdf FLUME-2365. Add DOAP file for Flume. Jun 13, 2014
pom.xml Add an HTTP sink Jan 24, 2017

README.md

Welcome to Apache Flume!

Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. The system is centrally managed and allows for intelligent dynamic management. It uses a simple extensible data model that allows for online analytic application.

The Apache Flume 1.x (NG) code line is a refactoring of the first generation Flume to solve certain known issues and limitations of the original design.

Apache Flume is open-sourced under the Apache Software Foundation License v2.0.

Documentation

Documentation is included in the binary distribution under the docs directory. In source form, it can be found in the flume-ng-doc directory.

The Flume 1.x guide and FAQ are available here:

Contact us!

Bug and Issue tracker.

Compiling Flume

Compiling Flume requires the following tools:

  • Oracle Java JDK 1.7
  • Apache Maven 3.x

Note: The Apache Flume build requires more memory than the default configuration. We recommend you set the following Maven options:

export MAVEN_OPTS="-Xms512m -Xmx1024m -XX:PermSize=256m -XX:MaxPermSize=512m"

To compile Flume and build a distribution tarball, run mvn install from the top level directory. The artifacts will be placed under flume-ng-dist/target/.