Skip to content

Commit

Permalink
[SPARK-1998] SparkFlumeEvent with body bigger than 1020 bytes are not…
Browse files Browse the repository at this point in the history
… re...

flume event sent to Spark will fail if the body is too large and numHeaders is greater than zero

Author: joyyoj <sunshch@gmail.com>

Closes apache#951 from joyyoj/master and squashes the following commits:

f4660c5 [joyyoj] [SPARK-1998] SparkFlumeEvent with body bigger than 1020 bytes are not read properly
  • Loading branch information
joyyoj authored and pwendell committed Jun 11, 2014
1 parent 1abbde0 commit 2966044
Showing 1 changed file with 2 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -71,12 +71,12 @@ class SparkFlumeEvent() extends Externalizable {
for (i <- 0 until numHeaders) {
val keyLength = in.readInt()
val keyBuff = new Array[Byte](keyLength)
in.read(keyBuff)
in.readFully(keyBuff)
val key : String = Utils.deserialize(keyBuff)

val valLength = in.readInt()
val valBuff = new Array[Byte](valLength)
in.read(valBuff)
in.readFully(valBuff)
val value : String = Utils.deserialize(valBuff)

headers.put(key, value)
Expand Down

0 comments on commit 2966044

Please sign in to comment.