Skip to content

Commit

Permalink
Rename readme to reflect use of markdown after addition of distribute…
Browse files Browse the repository at this point in the history
…d lzo indexer docs.
  • Loading branch information
kevinweil committed Jan 11, 2010
1 parent 4719c16 commit 221d68d
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions README → README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,11 @@ At this point, you should also be able to use the indexer to index lzo files in

- index it in-process via:

hadoop jar /path/to/your/hadoop-lzo.jar com.hadoop.compression.lzo.LzoIndexer big_file.lzo
hadoop jar /path/to/your/hadoop-lzo.jar com.hadoop.compression.lzo.LzoIndexer big_file.lzo

- index it in a map-reduce job via:

hadoop jar /path/to/your/hadoop-lzo.jar com.hadoop.compression.lzo.DistributedLzoIndexer big_file.lzo
hadoop jar /path/to/your/hadoop-lzo.jar com.hadoop.compression.lzo.DistributedLzoIndexer big_file.lzo

Either way, after 10-20 seconds there will be a file named big_file.lzo.index. The newly-created index file tells the LzoTextInputFormat's getSplits function how to break the LZO file into splits that can be decompressed and processed in parallel. Alternatively, if you specify a directory instead of a filename, both indexers will recursively walk the directory structure looking for .lzo files, indexing any that do not already have corresponding .lzo.index files.

Expand Down

0 comments on commit 221d68d

Please sign in to comment.