[FLINK-3640] Add support for SQL in DataSet programs #1862
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- add EnumerableToLogicalScan rule - in order to be able to mix TableAPI and SQL, we need our own copy of PlannerImpl
|
Awesome! Really cool to see that coming to Flink. Table table = tableEnv.fromDataSet(input);
tableEnv.registerTable("tab", table);
tableEnv.registerTable("tab1", table);
Table res = tableEnv.sql("SELECT COUNT(tab1.acount) AS acount, tab.word " +
"FROM tab, tab1 " +
"WHERE tab.word = tab1.word GROUP BY tab.word");
res = res.filter("acount > 2");Table res = tableEnv.sql("SELECT COUNT(acount) AS acount, word " +
"FROM (SELECT * FROM tab WHERE acount = 1 ) " +
"GROUP BY word"); |
| @@ -83,4 +84,17 @@ class AbstractTableEnvironment { | |||
| ) | |||
| TranslationContext.registerTable(dataSetTable, name) | |||
| } | |||
|
|
|||
| /** | |||
| * Execute a SQL query on a batch [[Table]]. | |||
There was a problem hiding this comment.
Keep the description of this method more generic as it will be the entry point for stream SQL queries as well.
| import scala.collection.JavaConverters._ | ||
|
|
||
| @RunWith(classOf[Parameterized]) | ||
| class AggregationsITCase( |
There was a problem hiding this comment.
Do we need each test for DataSet and Table? Wouldn't it be sufficient to test for Table and have one or two tests for DataSet?
|
Thanks for the PR. I had a few minor comments but otherwise it looks really good. There are a few follow up issues, IMO:
|
|
Thanks for the review @fhueske. I've addressed your comments :) |
|
|
||
| // initialize RelBuilder | ||
| frameworkConfig = Frameworks | ||
| .newConfigBuilder | ||
| .defaultSchema(tables) | ||
| .parserConfig(parserConfig) | ||
| .costFactory(new DataSetCostFactory) | ||
| .traitDefs(ConventionTraitDef.INSTANCE) | ||
| .programs(Programs.ofRules(FlinkRuleSets.DATASET_OPT_RULES)) |
There was a problem hiding this comment.
I think this line can be removed because we set the rules explicitly before calling the optimizer.
|
+1 to merge after resolving one last minor comment. |
|
Thanks, I will make the change and merge. |
fijolekProjects
pushed a commit
to fijolekProjects/flink
that referenced
this pull request
May 1, 2016
- add EnumerableToLogicalScan rule - in order to be able to mix TableAPI and SQL, we need our own copy of PlannerImpl - create a dummy RelNode in the reset() method, in order to retrieve the RelOptPlanner This closes apache#1862
hequn8128
pushed a commit
to hequn8128/flink
that referenced
this pull request
Jun 22, 2017
- add EnumerableToLogicalScan rule - in order to be able to mix TableAPI and SQL, we need our own copy of PlannerImpl - create a dummy RelNode in the reset() method, in order to retrieve the RelOptPlanner This closes apache#1862
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds basic support for batch SQL queries embedded in Table API programs.
In order to run a SQL query, a
DataSetorTableneeds to be registered in theTableEnvironmentand then the query is executed using thesqlmethod:The result of the
sqlmethod is aTablewhich can be used in subsequent Table API or SQL queries.