Skip to content


Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Elephant Twin LZO uses Elephant Twin to create LZO block indexes
branch: master

Merge pull request #1 from dvryaboy/master

Use generic block indexing code, fix up some tests.
latest commit c7178ab4fb
dvryaboy dvryaboy authored
Failed to load latest commit information.
src Pull out some of the common Block Indexing code.
.gitignore Initial Commit
LICENSE Initial Commit
NOTICE Initial Commit Initial Commit
pom.xml Pull out some of the common Block Indexing code.

elephant twin lzo

Elephant Twin is a framework for creating indexes in Hadoop.

Elephant Twin LZO uses ElephantTwin to create LZO block indexes. The resulting indexes can be used by Elephant Twin's BlockIndexedFileInputFormat to avoid scanning LZO blocks that do not contain the sought values of indexed columns. It can also be used by Elephant Twin's IndexedPigLoader to automatically apply filters found in Pig scripts, reducing the number of scanned data when reading LZO-compressed files.

This project is separate from Elephant Twin due to licensing restrictions due to the GPL.

LZO files are expected to be indexed using the indexing tools found in the Hadoop-LZO package.


First, ensure that native LZO libraries are installed.

Use the instructions at for getting this done.

After that, it should be a regular mvn install.


com.twitter.elephanttwin.lzo.retrieval.LZOBlockLevelIndexingJobs sets up and runs an indexing job over your lzo files.


Have a bug? Please create an issue here on GitHub!


For transparency and insight into our release cycle, releases will be numbered with the follow format:


And constructed with the following guidelines:

  • Breaking backwards compatibility bumps the major
  • New additions without breaking backwards compatibility bumps the minor
  • Bug fixes and misc changes bump the patch

For more information on semantic versioning, please visit


Major contributions from

  • Yu Xu
  • Jimmy Lin
  • Dmitriy Ryaboy

Other contributors can be found via git history (many thanks to all of them).


Copyright 2012 Twitter, Inc.

Licensed under the GNU General Public License. See

Something went wrong with that request. Please try again.