Skip to content

Commit

Permalink
Fixed ExpressionPayload accidentally pinning whole Avro record in m…
Browse files Browse the repository at this point in the history
…emory
  • Loading branch information
Alexey Kudinkin committed Dec 6, 2022
1 parent 58c53f8 commit b1c1b23
Showing 1 changed file with 8 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,14 @@ import scala.collection.mutable.ArrayBuffer
*
* If there is no condition match the record, ExpressionPayload will return
* a HoodieWriteHandle.IGNORE_RECORD, and the write handles will ignore this record.
*
* NOTE: Please note that, ctor parameter SHOULD NOT be used w/in the class body as
* otherwise Scala will instantiate them as fields making whole [[ExpressionPayload]]
* non-serializable. As an additional hedge, these are annotated as [[transient]] to
* prevent this from happening.
*/
class ExpressionPayload(record: GenericRecord,
orderingVal: Comparable[_])
class ExpressionPayload(@transient record: GenericRecord,
@transient orderingVal: Comparable[_])
extends DefaultHoodieRecordPayload(record, orderingVal) {

def this(recordOpt: HOption[GenericRecord]) {
Expand Down Expand Up @@ -167,7 +172,7 @@ class ExpressionPayload(record: GenericRecord,
lazy val row: InternalRow = getAvroDeserializerFor(avro.getSchema).deserialize(avro) match {
case Some(row) => row.asInstanceOf[InternalRow]
case None =>
logError(s"Failed to deserialize Avro record `${record.toString}` as Catalyst row")
logError(s"Failed to deserialize Avro record `${avro.toString}` as Catalyst row")
throw new HoodieException("Failed to deserialize Avro record as Catalyst row")
}
}
Expand Down

0 comments on commit b1c1b23

Please sign in to comment.