[SPARK-11251] Fix page size calculation in local mode

[andrewor14]

// My machine only has 8 cores
$ bin/spark-shell --master local[32]
scala> val df = sc.parallelize(Seq((1, 1), (2, 2))).toDF("a", "b")
scala> df.as("x").join(df.as("y"), $"x.a" === $"y.a").count()

Caused by: java.io.IOException: Unable to acquire 2097152 bytes of memory
    at org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorter.acquireNewPage(UnsafeExternalSorter.java:351)

[andrewor14]

@JoshRosen @rxin

[davies]

LGTM

[SparkQA]

Test build #44109 has finished for PR 9209 at commit d0fc050.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #1935 has finished for PR 9209 at commit d0fc050.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):\n * class BinaryClassificationEvaluator @Since(\"1.4.0\") (@Since(\"1.4.0\") override val uid: String)\n * class MulticlassClassificationEvaluator @Since(\"1.5.0\") (@Since(\"1.5.0\") override val uid: String)\n * final class RegressionEvaluator @Since(\"1.4.0\") (@Since(\"1.4.0\") override val uid: String)\n * case class ClassEncoder[T](\n * case class First(child: Expression, ignoreNullsExpr: Expression) extends DeclarativeAggregate\n * case class Last(child: Expression, ignoreNullsExpr: Expression) extends DeclarativeAggregate\n * case class First(\n * case class FirstFunction(\n * case class Last(\n * case class LastFunction(\n * case class CreateRow(children: Seq[Expression]) extends Expression\n * abstract class ColumnarIterator extends Iterator[InternalRow]\n * class SpecificColumnarIterator extends $\n

[SparkQA]

Test build #1936 has finished for PR 9209 at commit d0fc050.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[rxin]

Seems like there is a real failure?

[andrewor14]

What do you mean? It failed a flaky test LauncherBackendSuite. Let's retest this please

[SparkQA]

Test build #44172 has finished for PR 9209 at commit d0fc050.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[rxin]

I've merged this.