-
Notifications
You must be signed in to change notification settings - Fork 13.8k
[FLINK-29028][python][doc] Add the missing cache api in Python DataStream API #20618
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| .. versionadded:: 1.16.0 | ||
| """ | ||
| return DataStream(self._j_data_stream.cache()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess we may need to return CachedDataStream which has some specific API, e.g. invalidate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On top of that, we also introduce SideOutputDataStream which supports the cache method. I think we may need to introduce SideOutputDataStream in PyFlink and let get_side_output return SideOutputDataStream.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Sxnan I think we can reuse the cache method of DataStream, no need to introduce a new class SideOutputDataStream
| :return: The cached DataStream that can use in later job to reuse the cached intermediate | ||
| result. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Need also add close method in StreamExecutionEnvironment.
b0a8461 to
7bc0756
Compare
|
@HuangXingBo Thanks for your update! No more comments from my side. |
…am API This closes apache#20618.
What is the purpose of the change
This pull request will add the missing cache api in Python DataStream API
Brief change log
Does this pull request potentially affect one of the following parts:
@Public(Evolving): (no)Documentation