Skip to content
This repository has been archived by the owner on Sep 20, 2022. It is now read-only.

[HIVEMALL-30] Temporarily ignore a streaming test in Spark #28

Closed
wants to merge 1 commit into from
Closed

[HIVEMALL-30] Temporarily ignore a streaming test in Spark #28

wants to merge 1 commit into from

Conversation

maropu
Copy link
Member

@maropu maropu commented Jan 26, 2017

What changes were proposed in this pull request?

This test below fails sometimes (too flaky), so we temporarily ignore it.
The stacktrace of this failure is:

HivemallOpsWithFeatureSuite:
Exception in thread "broadcast-exchange-60" java.lang.OutOfMemoryError: Java heap space
	at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
	at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
	at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$4.apply(TorrentBroadcast.scala:231)
	at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$4.apply(TorrentBroadcast.scala:231)
	at org.apache.spark.util.io.ChunkedByteBufferOutputStream.allocateNewChunkIfNeeded(ChunkedByteBufferOutputStream.scala:78)
	at org.apache.spark.util.io.ChunkedByteBufferOutputStream.write(ChunkedByteBufferOutputStream.scala:65)
	at net.jpountz.lz4.LZ4BlockOutputStream.flushBufferedData(LZ4BlockOutputStream.java:205)
	at net.jpountz.lz4.LZ4BlockOutputStream.finish(LZ4BlockOutputStream.java:235)
	at net.jpountz.lz4.LZ4BlockOutputStream.close(LZ4BlockOutputStream.java:175)
	at java.io.ObjectOutputStream$BlockDataOutputStream.close(ObjectOutputStream.java:1827)
	at java.io.ObjectOutputStream.close(ObjectOutputStream.java:741)
	at org.apache.spark.serializer.JavaSerializationStream.close(JavaSerializer.scala:57)
	at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$blockifyObject$1.apply$mcV$sp(TorrentBroadcast.scala:238)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1296)
	at org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:237)
	at org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:107)
	at org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:86)
	at org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34)
	at org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:56)
	at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1370)

What type of PR is it?

Bug Fix

What is the Jira issue?

https://issues.apache.org/jira/browse/HIVEMALL-30

@maropu
Copy link
Member Author

maropu commented Jan 26, 2017

I turned off this test for now and keep looking into the root cause. If the cause found and fixed, I'll close the JIRA. So, I keep the JIRA ticket open.

@maropu maropu changed the title [HIVEMALL-30] Temporarily ignore a streaming test [HIVEMALL-30] Temporarily ignore a streaming test in Spark Jan 26, 2017
@coveralls
Copy link

Coverage Status

Changes Unknown when pulling b0feb79 on maropu:HIVEMALL-30 into ** on apache:master**.

1 similar comment
@coveralls
Copy link

coveralls commented Jan 26, 2017

Coverage Status

Changes Unknown when pulling b0feb79 on maropu:HIVEMALL-30 into ** on apache:master**.

@coveralls
Copy link

coveralls commented Jan 26, 2017

Coverage Status

Coverage increased (+0.01%) to 35.842% when pulling b0feb79 on maropu:HIVEMALL-30 into 79f92f4 on apache:master.

1 similar comment
@coveralls
Copy link

Coverage Status

Coverage increased (+0.01%) to 35.842% when pulling b0feb79 on maropu:HIVEMALL-30 into 79f92f4 on apache:master.

@maropu
Copy link
Member Author

maropu commented Jan 26, 2017

@myui okay to merge this?

@myui
Copy link
Member

myui commented Jan 26, 2017

@maropu LGTM. Please merge it.

@asfgit asfgit closed this in e86c8a0 Jan 26, 2017
@maropu
Copy link
Member Author

maropu commented Jan 26, 2017

Merged

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
3 participants