You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Transition will stand for interchanges between different in-memory data formats.
The enhancement includes:
Union current C2R / R2C / C2R2C / (future) C2C code together with centralized management for transitions.
Remove old rules for inserting C2R / R2C / C2R2C transitions.
Remove C2R / R2C creation APIs from SparkPlanExecApi.
Extensible convention API:
Convention will be bound to each individual plan operator to describe its supported output in-memory format.
Convention will consist of RowType and BatchType
BatchType is extensible with predefined API
When a new BatchType is defined, the relevant to-row / from-row / to-other-batch / from-other-batch transitions can be defined at the same time.
Insert transitions:
Remove old rules for inserting C2R / R2C / C2R2C transitions.
Remove code that will remain unused from PlanUtil.
Add a new rule to add transitions using the new facility. In the new rule, Gluten will find the cheapest defined transition for two adjacent plan operators.
The text was updated successfully, but these errors were encountered:
zhztheplayer
changed the title
[CORE] Rework C2R / R2C code with new transition facilities
[CORE] Rework planner C2R / R2C code with new transition facilities
May 16, 2024
Transition
will stand for interchanges between different in-memory data formats.The enhancement includes:
transition
s.convention
API:Convention
will be bound to each individual plan operator to describe its supported output in-memory format.Convention
will consist ofRowType
andBatchType
BatchType
is extensible with predefined APIBatchType
is defined, the relevantto-row
/from-row
/to-other-batch
/from-other-batch
transitions can be defined at the same time.PlanUtil
.The text was updated successfully, but these errors were encountered: