Replies: 1 comment
-
The comments on this issue may possibly be relevant (not 100% sure though) |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Through examining the current implementation of multi-column sort's spill-to-disk strategies, I find we are asking for more memory during spill, which I think is worth discussing:
During the spill,
Rows
are created for comparison efficiency for each in-memory RecordBatch. Considering why we spill in the first place, does thisRows
optimization increase the possibility of execution failure due to memory shortage?Please share your insights or corrections if I misunderstand any part of the code.
Beta Was this translation helpful? Give feedback.
All reactions