Skip to content

Conversation

LadyForest
Copy link
Contributor

@LadyForest LadyForest commented Mar 8, 2022

What is the purpose of the change

This PR aims to throw an explicit exception when querying a view with hints to improve user experience. Currently, the hints on view are swallowed quietly during the planning phase without any trace, and users complain that the hints don't work on view.

Brief changelog

  • Add check on ExpandingPreparingTable to throw a ValidationException
  • A minor fix for CliClientITCase to correct the assertion order for expected and actual

Verifying this change

This change added tests and can be verified as follows:

  • TableEnvironmentTest#testQueryViewWithHints
  • OptionHintsTest#testOptionsHintOnTableApiView and #testOptionsHintOnSQLView
  • Add test case in view.q, which can be verified by running CliClientITCase

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (yes / no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
  • The serializers: (yes / no / don't know)
  • The runtime per-record code paths (performance sensitive): (yes / no / don't know)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
  • The S3 file system connector: (yes / no / don't know)

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@flinkbot
Copy link
Collaborator

flinkbot commented Mar 8, 2022

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@LadyForest
Copy link
Contributor Author

@flinkbot run azure

@LadyForest
Copy link
Contributor Author

LadyForest commented Mar 8, 2022

@flinkbot run azure

https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=32676&view=logs&j=5c8e7682-d68f-54d1-16a2-a09310218a49&t=86f654fa-ab48-5c1a-25f4-7e7f6afb9bba

ChangelogPeriodicMaterializationITCase is unstable. My personal azure pipeline is successful.

@LadyForest
Copy link
Contributor Author

@flinkbot run azure

@LadyForest
Copy link
Contributor Author

LadyForest commented Mar 10, 2022

Hi, @JingsongLi, do you have time to take a look?

@JingsongLi JingsongLi self-requested a review March 15, 2022 14:19
@JingsongLi
Copy link
Contributor

Thanks @LadyForest for the contribution, can you add a case:

  • create view1 ....
  • create view2 select ... from view1 /** dynamic table hints .....*/;

Explain select view2.

def testOptionsHintOnTableApiView(): Unit = {
val view1 = util.tableEnv.sqlQuery("select * from t1 join t2 on t1.a = t2.d")
util.tableEnv.createTemporaryView("view1", view1)
// The table hints on view expect to be ignored.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears that the previous behavior was by design, so we need to ensure that it is fully corrected.

Copy link
Contributor Author

@LadyForest LadyForest Mar 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears that the previous behavior was by design.

Yes indeed. The hints are ignored quietly during planning phase. But it seems like our users are complaining the hints don't work. If we don't support hints on view, I think we'd better make it an explicit behavior. I'm afraid that the current design is somewhat not so much user-friendly. Beyond that, the current design is somewhat inconsistent, you can take a look at CatalogSourceTable#computeContextResolvedTable at L#144, it also throws exception when encounteredTableKind.VIEW. However, since the view expanding does not go through here, so this snippet of code does not get a chance to be called.

so we need to ensure that it is fully corrected

What do you mean by "ensure that it is fully corrected". Do you want to express we should ensure the exception is indeed thrown, or to ensure we don't throw exception when hints are imposed on table?

@LadyForest
Copy link
Contributor Author

Thanks @LadyForest for the contribution, can you add a case:
create view1 ....
create view2 select ... from view1 /** dynamic table hints .....*/;
Explain select view2.

Sure

@LadyForest
Copy link
Contributor Author

@flinkbot run azure

2 similar comments
@LadyForest
Copy link
Contributor Author

@flinkbot run azure

@LadyForest
Copy link
Contributor Author

@flinkbot run azure

… view

* Throw exception when expanding subquery
* Fix the assertion order in CliClientITCase
Copy link
Contributor

@JingsongLi JingsongLi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@JingsongLi JingsongLi merged commit e32c04c into apache:master Mar 22, 2022
JasonLeeCoding pushed a commit to JasonLeeCoding/flink that referenced this pull request May 27, 2022
zstraw pushed a commit to zstraw/flink that referenced this pull request Jul 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants