Browse files

FLUME29: Userdoc related to FLUME-29 - gzip compression

  • Loading branch information...
phunt committed Aug 9, 2010
1 parent 33dd870 commit 29040db76cc91fbaceb43896d942eeaf1127edc5
Showing with 14 additions and 0 deletions.
  1. +14 −0 src/docs/UserGuide/UsingOutput
@@ -208,6 +208,20 @@ Future versions of Hadoop will mitigate this problem by providing a flush/sync
operation for currently open HDFS files (patch is already slated for
Hadoop HDFS 0.21.x).
+== Gzip Compression for files written to HDFS.
+Flume now supports basic gzip compression for all log files that are written to HDFS.
+Compressed files are automatically suffixed with a .gz extension and follow the same naming format + directory structure as regularlog files.
+ <property>
+ <name>flume.collector.dfs.compress.gzip</name>
+ <value>true</value>
+ <description>Writes compressed output in gzip format to dfs. Setting value to false or empty writes uncompressed data to disk
+ </description>
+ </property>

0 comments on commit 29040db

Please sign in to comment.