Is your feature request related to a problem? Please describe.
Mark large non-serializable fields with @transient annotation across NativeExec classes to reduce task serialization size and memory usage.
Describe the solution you'd like
Introduce NativePartition[P] wrapper to carry original partition payloads through NativeRDD without re-indexing into the partitions array at execution time, which avoids serializing the full partitions array.
Describe alternatives you've considered
Additional context
Is your feature request related to a problem? Please describe.
Mark large non-serializable fields with
@transientannotation across NativeExec classes to reduce task serialization size and memory usage.Describe the solution you'd like
Introduce NativePartition[P] wrapper to carry original partition payloads through NativeRDD without re-indexing into the partitions array at execution time, which avoids serializing the full partitions array.
Describe alternatives you've considered
Additional context