Skip to content

Conversation

@mustafasrepo
Copy link
Contributor

@mustafasrepo mustafasrepo commented Aug 8, 2023

Which issue does this PR close?

Closes #.

Rationale for this change

At the projection of the source, we keep track of ordering as long as they are used in the query.
Consider table below

CREATE EXTERNAL TABLE multiple_ordered_table (
  a0 INTEGER,
  a INTEGER,
  b INTEGER,
  c INTEGER,
  d INTEGER
)
STORED AS CSV
WITH HEADER ROW
WITH ORDER (a ASC, b ASC)
WITH ORDER (c ASC)
LOCATION 'tests/data/window_2.csv';

which satisfies ordering a ASC, b ASC and c ASC.
If we run the query below on this table.

SELECT c
FROM multiple_ordered_table
ORDER BY c ASC;

since, only column c is used. We will not keep track of ordering a ASC, b ASC. However, during projection of orderings
empty vectors are still added to output ordering. Since, first entry in among alternative orderings is output_ordering. Output ordering of the CsvExec is empty.
In this PR, we add emptiness check, with this check we no longer store empty orderings(after projection) at the source. After this PR output ordering of the CsvExec is no longer empty.

What changes are included in this PR?

Are these changes tested?

Yes, new tests are added.

Are there any user-facing changes?

@github-actions github-actions bot added core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) labels Aug 8, 2023
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mustafasrepo

}
all_orderings.push(new_ordering);
// do not push empty entries
// otherwise we may have `Some(vec![])` at the output ordering.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@alamb alamb merged commit 43f182f into apache:main Aug 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants