Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

Explicitly disable map speculative execution, since LzoIndexOutputFor…

…mat doesn't work with it.

This could be fixed, but would require writing a proper OutputCommiter. Patches welcome.

Bumps version to 0.4.2
  • Loading branch information...
commit 1053d83c2e59f90a16a35c040ec8711d8a5a649f 1 parent 98c763d
@toddlipcon toddlipcon authored
View
2  build.xml
@@ -28,7 +28,7 @@
<property name="Name" value="Hadoop GPL Compression"/>
<property name="name" value="hadoop-lzo"/>
- <property name="version" value="0.4.1"/>
+ <property name="version" value="0.4.2"/>
<property name="final.name" value="${name}-${version}"/>
<property name="year" value="2008"/>
View
5 src/java/com/hadoop/compression/lzo/DistributedLzoIndexer.java
@@ -93,6 +93,11 @@ public int run(String[] args) throws Exception {
job.setOutputKeyClass(Path.class);
job.setOutputValueClass(LongWritable.class);
+ // The LzoIndexOutputFormat doesn't currently work with speculative execution.
+ // Patches welcome.
+ job.getConfiguration().setBoolean(
+ "mapred.map.tasks.speculative.execution", false);
+
job.setJarByClass(DistributedLzoIndexer.class);
job.setInputFormatClass(LzoSplitInputFormat.class);
job.setOutputFormatClass(LzoIndexOutputFormat.class);
Please sign in to comment.
Something went wrong with that request. Please try again.