SPARK-11348 Replace addOnCompleteCallback with addTaskCompletionListe…

[tedyu]

…ner() in UnsafeExternalSorter

When practicing the command from SPARK-11318, I got the following:

[WARNING] /home/hbase/spark/core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java:[141,15] [deprecation]                                      addOnCompleteCallback(Function0<BoxedUnit>) in TaskContext has been deprecated

addOnCompleteCallback should be replaced with addTaskCompletionListener()

[tedyu]

Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress https://github.com/apache/spark.git +refs/pull/9356/*:refs/remotes/origin/pr/9356/*" returned status code 143:
stdout: 
stderr: error: RPC failed; result=18, HTTP code = 200

Not sure what caused the above failure

[SparkQA]

Test build #44610 has finished for PR 9356 at commit 4efee6b.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[tedyu]

[info] StandaloneDynamicAllocationSuite:
[info] - dynamic allocation default behavior (1 second, 162 milliseconds)
[info] - dynamic allocation with max cores <= cores per worker (365 milliseconds)
[info] - dynamic allocation with max cores > cores per worker (671 milliseconds)
[info] - dynamic allocation with cores per executor (742 milliseconds)
[info] - dynamic allocation with cores per executor AND max cores (1 second, 186 milliseconds)
[info] - kill the same executor twice (SPARK-9795) (150 milliseconds)
[info] - the pending replacement executors should not be lost (SPARK-10515) (557 milliseconds)
Exception in thread "Thread-45" Exception in thread "Thread-41" java.io.EOFException
    at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2598)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1318)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
    at org.scalatest.tools.Framework$ScalaTestRunner$Skeleton$1$React.react(Framework.scala:945)
    at org.scalatest.tools.Framework$ScalaTestRunner$Skeleton$1.run(Framework.scala:934)
    at java.lang.Thread.run(Thread.java:745)
java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:196)
    at java.net.SocketInputStream.read(SocketInputStream.java:122)
    at java.net.SocketInputStream.read(SocketInputStream.java:210)
    at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2293)
    at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2586)
    at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2596)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1318)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
    at sbt.React.react(ForkTests.scala:114)
    at sbt.ForkTests$$anonfun$mainTestTask$1$Acceptor$2$.run(ForkTests.scala:74)
    at java.lang.Thread.run(Thread.java:745)
[info] ScalaTest
[info] Run completed in 2 minutes, 50 seconds.
[info] Total number of tests run: 107
[info] Suites: completed 9, aborted 0
[info] Tests: succeeded 107, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[error] Error: Total 0, Failed 0, Errors 0, Passed 0
[error] Error during tests:
...
[error] (core/test:test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 3801 s, completed Oct 29, 2015 10:28:05 AM

I don't think the above was related to my change

[SparkQA]

Test build #44626 has finished for PR 9356 at commit 57b92b5.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

This is already covered by 56419cf Do you mind closing this PR?

[tedyu]

Indeed - Davies is fast