Skip to content

Conversation

@linhr
Copy link
Contributor

@linhr linhr commented Jun 23, 2024

Which issue does this PR close?

Closes #11072.

Rationale for this change

This PR changes the wildcard qualifier type to TableReference. This avoids an implicit SQL parsing during logical planning, and makes the API more consistent.

What changes are included in this PR?

The PR changes the wildcard qualifier type for Expr::Wildcard and updates the .proto file.

Are these changes tested?

The changes are covered by existing tests.

Are there any user-facing changes?

Yes, this PR contains breaking changes to the public API.

@github-actions github-actions bot added sql SQL Planner logical-expr Logical plan and expressions labels Jun 23, 2024
Comment on lines 372 to 373
string qualifier = 1;
string qualifier = 1 [deprecated = true];
TableReference relation = 2;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Theoretically this should be a oneof, but I'm not sure if adding oneof will alter the pbjson serialization. Since we are deprecating the field anyway, I decided to handle the disambiguation in from_proto.rs and to_proto.rs instead.

Also, I'm not sure if we are allowed to make breaking changes to protobuf here. If we do, I can simply change the type of qualifier to make the code simpler.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I'm not sure if we are allowed to make breaking changes to protobuf here. If we do, I can simply change the type of qualifier to make the code simpler.

At the moment, we don't try to maintain any backwards compatibility, thus I think we should opt for simpler code

//! # Version Compatibility
//!
//! The serialized form are not guaranteed to be compatible across
//! DataFusion versions. A plan serialized with one version of DataFusion
//! may not be able to deserialized with a different version.
//!

@alamb alamb changed the title Change wildcard qualifier type Change wildcard qualifier type from String to TableReference Jun 23, 2024
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @linhr -- this change makes sense to me

I think the protobuf code could be made simpler as I mentioned, but otherwise this seems like a good cleanup to me. Thank you

Comment on lines 372 to 373
string qualifier = 1;
string qualifier = 1 [deprecated = true];
TableReference relation = 2;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I'm not sure if we are allowed to make breaking changes to protobuf here. If we do, I can simply change the type of qualifier to make the code simpler.

At the moment, we don't try to maintain any backwards compatibility, thus I think we should opt for simpler code

//! # Version Compatibility
//!
//! The serialized form are not guaranteed to be compatible across
//! DataFusion versions. A plan serialized with one version of DataFusion
//! may not be able to deserialized with a different version.
//!

@linhr
Copy link
Contributor Author

linhr commented Jun 24, 2024

Thank you @alamb ! I've updated the protobuf code as suggested.

@linhr linhr requested a review from alamb June 24, 2024 06:34
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again @linhr

@alamb alamb merged commit a202a01 into apache:main Jun 24, 2024
@linhr linhr deleted the wildcard-qualifier-type branch June 25, 2024 01:02
findepi pushed a commit to findepi/datafusion that referenced this pull request Jul 16, 2024
…che#11073)

* Change wildcard qualifier type

* Update protobuf

* Minor update
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logical-expr Logical plan and expressions sql SQL Planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expr::Wildcard should use TableReference as the qualifier type

2 participants