Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-10676][table] Add 'as' method for OverWindowWithOrderBy #6949

Closed
wants to merge 2 commits into from

Conversation

hequn8128
Copy link
Contributor

What is the purpose of the change

The preceding clause of OVER Window in the traditional database is optional. The default is UNBOUNDED. So we can add the as method to OverWindowWithOrderBy. This way OVER Window is written more easily. e.g.:
.window(Over partitionBy 'c orderBy 'proctime preceding UNBOUNDED_ROW as 'w)
Can be simplified as follows:
.window(Over partitionBy 'c orderBy 'proctime as 'w)

SQL has already supported such feature.

Brief change log

  • Add as function for OverWindowWithOrderBy both for scala and java.
  • Add tests to check result plan.

Verifying this change

This change added tests and can be verified as follows:

  • Added tests that validate result plan, such as tests in OverWindowTest and OverWindowStringExpressionTest.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (yes)
  • If yes, how is the feature documented? (docs)

Copy link
Member

@sunjincheng121 sunjincheng121 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @hequn8128 Thanks for the PR.
The PR LGTM, except for a few comments I left.
Best,
Jincheng

@@ -18,7 +18,8 @@

package org.apache.flink.table.api.java

import org.apache.flink.table.api.{TumbleWithSize, OverWindowWithPreceding, SlideWithSize, SessionWithGap}
import org.apache.flink.table.api.scala.{CURRENT_RANGE, UNBOUNDED_RANGE}
import org.apache.flink.table.api._
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think using import org.apache.flink.table.api.{OverWindow, TumbleWithSize, OverWindowWithPreceding, SlideWithSize, SessionWithGap} is better than using wildcard, because there are not many classes imported here.What do you think?

<td>
<p>Defines the interval of rows that are included in the window and precede the current row. The interval can either be specified as time or row-count interval.</p>

<p><a href="tableApi.html#bounded-over-windows">Bounded over windows</a> are specified with the size of the interval, e.g., <code>10.minutes</code> for a time interval or <code>10.rows</code> for a row-count interval.</p>

<p><a href="tableApi.html#unbounded-over-windows">Unbounded over windows</a> are specified using a constant, i.e., <code>UNBOUNDED_RANGE</code> for a time interval or <code>UNBOUNDED_ROW</code> for a row-count interval. Unbounded over windows start with the first row of a partition.</p>

<p>If the <code>preceding</code> and <code>following</code> clause both are omitted, RANGE UNBOUNDED PRECEDING AND CURRENT ROW is used as default for window.</p>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UNBOUNDED_RANGE and CURRENT_RANGE appear in pairs,So the following description is recommended:
If the preceding clause is omitted, UNBOUNDED_RANGE and CURRENT_RANGE is used as default for window. What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sense!

@@ -18,8 +18,8 @@

package org.apache.flink.table.api.scala

import org.apache.flink.table.api.{TumbleWithSize, OverWindowWithPreceding, SlideWithSize, SessionWithGap}
import org.apache.flink.table.expressions.Expression
import org.apache.flink.table.api._
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as java/windows.scala comments.

@hequn8128
Copy link
Contributor Author

Hi @sunjincheng121 , thanks a lot for your view. I think all of the suggestions are good! I have addressed the comments and updated the pr.

@sunjincheng121
Copy link
Member

sunjincheng121 commented Nov 7, 2018

@hequn8128 Thanks for the updated!
+1 to merged.

Thanks,
Jincheng

@sunjincheng121
Copy link
Member

CI find JobManagerHAProcessFailureRecoveryITCase.testDispatcherProcessFailure:331 » IllegalArgument test fail which looks no relation with this PR. I'll check it, then merge this PR.

@sunjincheng121
Copy link
Member

After the rerun, the error disappeared. no specific reasons are found, so created a JIRA. FLINK-10819, and will continue to pay attention.

Merging this PR...

@asfgit asfgit closed this in 483507a Nov 8, 2018
@hequn8128
Copy link
Contributor Author

Thanks a lot for the review and merging!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants