[SPARK-16003] SerializationDebugger runs into infinite loop

[ericl]

What changes were proposed in this pull request?

This fixes SerializationDebugger to not recurse forever when writeReplace returns an object of the same class, which is the case for at least the SQLMetrics class.

See also the OpenJDK unit tests on the behavior of recursive writeReplace():
https://github.com/openjdk-mirror/jdk7u-jdk/blob/f4d80957e89a19a29bb9f9807d2a28351ed7f7df/test/java/io/Serializable/nestedReplace/NestedReplace.java

cc @davies @cloud-fan

How was this patch tested?

Unit tests for SerializationDebugger.

[SparkQA]

Test build #60956 has finished for PR 13814 at commit 3237c27.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[cloud-fan]

LGTM

[davies]

What's the error message looks like in the case of unserializable object (for example, Iterator in scala 2.10)?

[andrewor14]

LGTM

[ericl]

@davies the message looks ok to me:

Caused by: java.io.NotSerializableException: scala.collection.Iterator$$anon$13
Serialization stack:

    at org.apache.spark.serializer.SerializationDebugger$.improveException(SerializationDebugger.scala:40)
    at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:46)
    at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:100)
    at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:295)
    ... 44 more

[davies]

Looks good, merging this into master and 2.0 branch.