Skip to content

Commit

Permalink
Added info log message when writing "direct" to a filesystem, bypassi…
Browse files Browse the repository at this point in the history
…ng the temporary folder removing the need to rename the output file to its target location.
  • Loading branch information
cwensel committed Jul 8, 2011
1 parent 0c0509c commit 114494c
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 1 deletion.
3 changes: 3 additions & 0 deletions CHANGES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ Cascading Change Log

1.2.4 [unreleased]

Added info log message when writing "direct" to a filesystem, bypassing the temporary folder removing the need to
rename the output file to its target location.

Fixed bug where if all paths that match a glob pattern are empty, an exception is not thrown causing Hadoop to throw
a java.lang.ArrayIndexOutOfBoundsException.

Expand Down
7 changes: 6 additions & 1 deletion src/core/cascading/tap/hadoop/Hadoop18TapUtil.java
Original file line number Diff line number Diff line change
Expand Up @@ -341,7 +341,12 @@ private static boolean writeDirectlyToWorkingPath( JobConf conf, Path path )
if( fs == null )
return false;

return conf.getBoolean( "mapred.output.direct." + fs.getClass().getSimpleName(), false );
boolean result = conf.getBoolean( "mapred.output.direct." + fs.getClass().getSimpleName(), false );

if( result )
LOG.info( "output direct is enabled for this fs: " + fs.getName() );

return result;
}

}

0 comments on commit 114494c

Please sign in to comment.