[FLINK-14067] Decouple PlanExecutor from JSON plan generation #9690
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
Thanks @aljoscha ! I will have a look |
Collaborator
|
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community Automated ChecksLast check on commit 0e68c41 (Wed Oct 16 08:18:57 UTC 2019) Warnings:
Mention the bot in a comment to re-run the automated checks. Review Progress
Please see the Pull Request Review Guide for a full explanation of the review process. DetailsThe Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commandsThe @flinkbot bot supports the following commands:
|
aljoscha
changed the title
kl0u
approved these changes
Sep 17, 2019
flink-java/src/main/java/org/apache/flink/api/java/ExecutionPlanUtil.java
Outdated
Show resolved
Hide resolved
| public String getExecutionPlan() throws Exception { | ||
| Plan plan = createProgramPlan("unnamed job"); | ||
|
|
||
| OptimizedPlan op = ClusterClient.getOptimizedPlan(client.compiler, plan, getParallelism()); |
Contributor
There was a problem hiding this comment.
I think that now we can make the client.compiler field private to avoid leaking information about internal state.
Contributor
There was a problem hiding this comment.
And maybe other ClusterClient fields can become private as well.
| * @throws Exception Thrown, if the compiler could not be instantiated. | ||
| */ | ||
| public abstract String getExecutionPlan() throws Exception; | ||
| public String getExecutionPlan() throws Exception { |
Contributor
There was a problem hiding this comment.
I think you can remove the Exception.
Contributor
Author
There was a problem hiding this comment.
I didn't want to remove it here because it is a @Public class.
flink-java/src/main/java/org/apache/flink/api/java/ExecutionEnvironment.java
Show resolved
Hide resolved
flink-java/src/main/java/org/apache/flink/api/java/ExecutionPlanUtil.java
Show resolved
Hide resolved
flink-java/src/main/java/org/apache/flink/api/java/ExecutionPlanUtil.java
Show resolved
Hide resolved
…in root class Before, each subclass had a slightly different way of getting the execution plan (as JSON). Now, we factor that part out into a utility and use that in the ExecutionEnvironment root class. This does mean, that we don't take into account special information that a cluster client or some other environment might have for plan generation. Also, we can't remove the throws Exception from getExecutionPlan() because it is a @public method.
We do this to get rid of the dependency on PlanExecutor for generating the JSON plan. The executor should not be concerned with printing JSON plans and this will simplify future executor work.
aljoscha
force-pushed
the
decouple-plan-preview
branch
from
15a0e84 to
0e68c41
Compare
Contributor
Author
|
merged |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
{{PlanExecutor}} has a method {{getOptimizerPlanAsJSON()}} that is used by DataSet environments to get a JSON version of the execution plan. To ease future work and to make it more maintainable we should get rid of that method and instead have a dedicated utility for generating JSON plans that the environments can use.
(The only reason this method is on the executor is because only {{flink-clients}} via {{flink-optimizer}} has the required components to derive a JSON plan.)
Brief change log
Please look at the list of commits for a changelog, each commit is an isolated refactoring on the path to the final result.
Verifying this change
ExecutionPlanUtilDocumentation