Skip to content
Permalink
Browse files

[HOT-FIX] Fix SparkOutOfMemoryError: Unable to acquire 262144 bytes o…

…f memory, got 224631

## What changes were proposed in this pull request?

https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88263/testReport
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88260/testReport
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88257/testReport
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88224/testReport

These tests all failed:
```
org.apache.spark.memory.SparkOutOfMemoryError:  Unable to acquire 262144 bytes of memory, got 224631
at org.apache.spark.memory.MemoryConsumer.throwOom(MemoryConsumer.java:157)
at org.apache.spark.memory.MemoryConsumer.allocateArray(MemoryConsumer.java:98)
at org.apache.spark.unsafe.map.BytesToBytesMap.allocate(BytesToBytesMap.java:787)
at org.apache.spark.unsafe.map.BytesToBytesMap.<init>(BytesToBytesMap.java:204)
at org.apache.spark.unsafe.map.BytesToBytesMap.<init>(BytesToBytesMap.java:219)
...
```

This PR ignore this test.

## How was this patch tested?

N/A

Author: Yuming Wang <yumwang@ebay.com>

Closes #20835 from wangyum/SPARK-23598.

(cherry picked from commit 15c3c98)
Signed-off-by: Herman van Hovell <hvanhovell@databricks.com>
  • Loading branch information...
wangyum authored and hvanhovell committed Mar 15, 2018
1 parent 57026a1 commit 2fd7acabf8d55789662d52d94bd30f84b05a577a
@@ -310,7 +310,7 @@ class WholeStageCodegenSuite extends QueryTest with SharedSQLContext {
}
}

test("SPARK-23598: Codegen working for lots of aggregation operations without runtime errors") {
ignore("SPARK-23598: Codegen working for lots of aggregation operations without runtime errors") {
withSQLConf(SQLConf.SHUFFLE_PARTITIONS.key -> "1") {
var df = Seq((8, "bat"), (15, "mouse"), (5, "horse")).toDF("age", "name")
for (i <- 0 until 70) {

0 comments on commit 2fd7aca

Please sign in to comment.
You can’t perform that action at this time.