Skip to content

Conversation

@doki23
Copy link
Contributor

@doki23 doki23 commented Mar 14, 2022

Which issue does this PR close?

Closes #1994.

What changes are included in this PR?

Add enum Expr::QualifiedWildcard
Add function: expand_qualified_wildcard
Add some tests

@github-actions github-actions bot added datafusion sql SQL Planner labels Mar 14, 2022
@doki23 doki23 changed the title Qualified wildcard Qualified wildcard(WIP) Mar 15, 2022
@doki23 doki23 marked this pull request as draft March 15, 2022 02:33
@doki23 doki23 marked this pull request as ready for review March 15, 2022 02:54
@doki23 doki23 changed the title Qualified wildcard(WIP) Qualified wildcard Mar 15, 2022
@doki23
Copy link
Contributor Author

doki23 commented Mar 17, 2022

Would you please find time to review this pr?
@alamb @houqp @yjshen @xudong963

@xudong963
Copy link
Member

Thanks @doki23, will take a look tomorrow :)

@xudong963 xudong963 added the enhancement New feature or request label Mar 17, 2022
Expr::Wildcard => Err(DataFusionError::Internal(
"Create name does not support wildcard".to_string(),
)),
Expr::QualifiedWildcard { .. } => Err(DataFusionError::Internal(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can create a name for QualifiedWildcard?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree including the name of the qualifier in the message might be helpful

Expr::Wildcard => Err(DataFusionError::Internal(
"Create physical name does not support wildcard".to_string(),
)),
Expr::QualifiedWildcard { .. } => Err(DataFusionError::Internal(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@doki23 doki23 marked this pull request as draft March 20, 2022 01:49
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is looking very nice -- thank you @doki23

Expr::Wildcard => Err(DataFusionError::Internal(
"Create name does not support wildcard".to_string(),
)),
Expr::QualifiedWildcard { .. } => Err(DataFusionError::Internal(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree including the name of the qualifier in the message might be helpful

@alamb
Copy link
Contributor

alamb commented Mar 21, 2022

Upon further thought, I wonder if it would make sense to expand out all qualified wildcards in the Sql -> LogicalPlan planner rather than having an Expr 🤔 and have to error when we encountered it.

Though I suppose if we did that then we couldn't support wild cards in the DataFrame API 🤔

@doki23 doki23 marked this pull request as ready for review March 22, 2022 13:36
@doki23
Copy link
Contributor Author

doki23 commented Mar 22, 2022

I think it's ready for review

@doki23
Copy link
Contributor Author

doki23 commented Mar 22, 2022

Upon further thought, I wonder if it would make sense to expand out all qualified wildcards in the Sql -> LogicalPlan planner rather than having an Expr 🤔 and have to error when we encountered it.

Though I suppose if we did that then we couldn't support wild cards in the DataFrame API 🤔

We should support wildcard in any kinds of api, including DataFrame.

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it looks good to me -- thanks @doki23 !

Ok(_) => panic!("unexpected OK"),
Err(err) => assert_eq!(
err.to_string(),
"Error during planning: Invalid qualifier agg"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

},
/// Represents a reference to all fields in a schema.
Wildcard,
/// Represents a reference to all fields in a specific schema.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally have a slight preference for a single Expr::Wildcard with optional qualifier

  Wildcard { qualifier: Option<String>}

but that is a personal preference and there is nothing wrong with this approach

Copy link
Member

@xudong963 xudong963 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xudong963 xudong963 requested a review from houqp March 23, 2022 13:55
@alamb
Copy link
Contributor

alamb commented Mar 23, 2022

Thanks again @doki23

@alamb alamb merged commit 634252b into apache:master Mar 23, 2022
@doki23
Copy link
Contributor Author

doki23 commented Mar 24, 2022

Thanks @xudong963 @alamb ! ❤️

@doki23 doki23 deleted the qualified_wildcard branch March 24, 2023 03:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request sql SQL Planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement qualified wildcard

3 participants