[compiler] ensure CDA result Array[Array[Byte]] can be GCed after dec…#13011
[compiler] ensure CDA result Array[Array[Byte]] can be GCed after dec…#13011danking merged 3 commits intohail-is:mainfrom
Conversation
|
|
||
| val encRes = cb.newLocal[Array[Array[Byte]]]("encRes") | ||
| val len = mb.newLocal[Int]("cda_result_length") | ||
| val ib = mb.newLocal[InputBuffer]("decode_ib") |
There was a problem hiding this comment.
why sometimes class builder sometimes method builder?
There was a problem hiding this comment.
More direct: How do you pick when to use mb vs cb?
There was a problem hiding this comment.
These are aliased. Rewrote to use the more idiomatic "memoize" now and avoid explicit variable declarations.
The rule is basically to use the lowest-level builder that has the functionality you want. Module > Class > Method > Code, but codeBuilder doesn't duplicate the full functionality of everything above. So to create a field, use methodBuilder (it exists on MB and ClassB, not codeB).
| .asBaseStruct | ||
| eltTupled.loadField(cb, 0) | ||
| } | ||
| cb.assign(encRes, Code._null) |
There was a problem hiding this comment.
Is it not possible to also free each block after decoding it? As written we'd need twice the RAM to read a given result array, I think.
| }) | ||
|
|
||
| val encRes = cb.newLocal[Array[Array[Byte]]]("encRes") | ||
| cb.assign(encRes, spark.invoke[BackendContext, HailClassLoader, FS, String, Array[Array[Byte]], Array[Byte], String, Option[TableStageDependency], Array[Array[Byte]]]( |
There was a problem hiding this comment.
Approved, but is it possible to use memoize here as well or is this special in some way?
2 participants
…oding