Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Fix show problem by changing TableOrderBy to accept keyed tables. #5172
It wasn't scanning the full dataset anymore, but:
table.head().flatten() was generating a TableOrderBy(TableKeyBy(TableHead)).
There was no way to remove this node, even if the table was already keyed by the sort fields, so we ended up doing an extra scan and possibly shuffle.
This change simplifies the whole thing, and emits the correct IR from the beginning