From 6c323fdc1a0013938d09b09b2e16061910a92c97 Mon Sep 17 00:00:00 2001
From: Javeme <javaloveme@gmail.com>
Date: Fri, 30 Dec 2016 19:39:20 +0800
Subject: [PATCH] MAPREDUCE-6827. Failed to traverse Iterable values the second
 time in reduce() method

The following code is a reduce() method (of WordCount):

	public static class WcReducer extends Reducer<Text, IntWritable, Text, IntWritable> {

		@Override
		protected void reduce(Text key, Iterable<IntWritable> values, Context context)
				throws IOException, InterruptedException {

			// print some logs
			List<String> vals = new LinkedList<>();
			for(IntWritable i : values) {
				vals.add(i.toString());
			}
			System.out.println(String.format(">>>> reduce(%s, [%s])",
					key, String.join(", ", vals)));

			// sum of values
			int sum = 0;
			for(IntWritable i : values) {
				sum += i.get();
			}
			System.out.println(String.format(">>>> reduced(%s, %s)",
					key, sum));

			context.write(key, new IntWritable(sum));
		}
	}

After running it, we got the result that all sums were zero!

After debugging, it was found that the second foreach-loop was not executed, and the root cause was the returned value of Iterable.iterator(), it returned the same instance in the two calls by foreach-loop. In general, Iterable.iterator() should return a new instance in each call, such as ArrayList.iterator(). This patch fixed the bug.

Signed-off-by: Javeme <javaloveme@gmail.com>
---
 .../org/apache/hadoop/mapreduce/task/ReduceContextImpl.java    | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/ReduceContextImpl.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/ReduceContextImpl.java
index 43ce837092b84d..208d428b2bef6d 100644
--- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/ReduceContextImpl.java
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/ReduceContextImpl.java
@@ -348,10 +348,9 @@ private void writeFirstKeyValueBytes(DataOutputStream out)
   }
 
   protected class ValueIterable implements Iterable<VALUEIN> {
-    private ValueIterator iterator = new ValueIterator();
     @Override
     public Iterator<VALUEIN> iterator() {
-      return iterator;
+      return new ValueIterator();
     } 
   }