[SPARK-9266] Prevent "managed memory leak detected" exception from masking original exception

[JoshRosen]

When a task fails with an exception and also fails to properly clean up its managed memory, the spark.unsafe.exceptionOnMemoryLeak memory leak detection mechanism's exceptions will mask the original exception that caused the task to fail. We should throw the memory leak exception only if no other exception occurred.

[JoshRosen]

Jenkins, retest this please.

[rxin]

LGTM

[SparkQA]

Test build #38129 has finished for PR 7603 at commit c1f0167.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class UnresolvedFunction(
  • case class Average(child: Expression) extends AlgebraicAggregate
  • case class Count(child: Expression) extends AlgebraicAggregate
  • case class First(child: Expression) extends AlgebraicAggregate
  • case class Last(child: Expression) extends AlgebraicAggregate
  • case class Max(child: Expression) extends AlgebraicAggregate
  • case class Min(child: Expression) extends AlgebraicAggregate
  • case class Sum(child: Expression) extends AlgebraicAggregate
  • abstract class AlgebraicAggregate extends AggregateFunction2 with Serializable
  • implicit class RichAttribute(a: AttributeReference)
  • trait AggregateExpression1 extends AggregateExpression
  • trait PartialAggregate1 extends AggregateExpression1
  • case class Min(child: Expression) extends UnaryExpression with PartialAggregate1
  • case class MinFunction(expr: Expression, base: AggregateExpression1) extends AggregateFunction1
  • case class Max(child: Expression) extends UnaryExpression with PartialAggregate1
  • case class MaxFunction(expr: Expression, base: AggregateExpression1) extends AggregateFunction1
  • case class Count(child: Expression) extends UnaryExpression with PartialAggregate1
  • case class CountFunction(expr: Expression, base: AggregateExpression1) extends AggregateFunction1
  • case class CountDistinct(expressions: Seq[Expression]) extends PartialAggregate1
  • case class CollectHashSet(expressions: Seq[Expression]) extends AggregateExpression1
  • case class CombineSetsAndCount(inputSet: Expression) extends AggregateExpression1
  • case class Average(child: Expression) extends UnaryExpression with PartialAggregate1
  • case class AverageFunction(expr: Expression, base: AggregateExpression1)
  • case class Sum(child: Expression) extends UnaryExpression with PartialAggregate1
  • case class SumFunction(expr: Expression, base: AggregateExpression1) extends AggregateFunction1
  • case class CombineSum(child: Expression) extends AggregateExpression1
  • case class CombineSumFunction(expr: Expression, base: AggregateExpression1)
  • case class SumDistinct(child: Expression) extends UnaryExpression with PartialAggregate1
  • case class SumDistinctFunction(expr: Expression, base: AggregateExpression1)
  • case class CombineSetsAndSum(inputSet: Expression, base: Expression) extends AggregateExpression1
  • case class First(child: Expression) extends UnaryExpression with PartialAggregate1
  • case class FirstFunction(expr: Expression, base: AggregateExpression1) extends AggregateFunction1
  • case class Last(child: Expression) extends UnaryExpression with PartialAggregate1
  • case class LastFunction(expr: Expression, base: AggregateExpression1) extends AggregateFunction1
  • case class CreateArray(children: Seq[Expression]) extends Expression
  • case class CreateStruct(children: Seq[Expression]) extends Expression
  • case class CreateNamedStruct(children: Seq[Expression]) extends Expression
  • case class FormatString(children: Expression*) extends Expression with ImplicitCastInputTypes
  • case class Aggregate2Sort(
  • case class FinalAndCompleteAggregate2Sort(
  • class GroupingIterator(
  • class PartialSortAggregationIterator(
  • class PartialMergeSortAggregationIterator(
  • class FinalSortAggregationIterator(
  • class FinalAndCompleteSortAggregationIterator(
  • abstract class UserDefinedAggregateFunction extends Serializable
  • case class ScalaUDAF(

[JoshRosen]

Looks like the failed test might be an unrelated problem in master, but let's retest again just to be sure...

[JoshRosen]

Jenkins, retest this please.

[SparkQA]

Test build #38149 has finished for PR 7603 at commit c1f0167.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #38172 has finished for PR 7603 at commit c268cb5.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[rxin]

I've merged this.