Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-4130] Add tests for FlinkJobServerDriver #6703

Merged
merged 1 commit into from Oct 17, 2018

Conversation

mxm
Copy link
Contributor

@mxm mxm commented Oct 16, 2018

This adds a few test cases for FlinkJobServerDriver. It also changes the default
host from empty host to localhost.

Backported from master (cfaa01a)

CC @tweise @angoenka

Post-Commit Tests Status (on master branch)

Lang SDK Apex Dataflow Flink Gearpump Samza Spark
Go Build Status --- --- --- --- --- ---
Java Build Status Build Status Build Status Build Status Build Status Build Status Build Status
Python Build Status --- Build Status
Build Status
Build Status --- --- ---

@mxm mxm requested a review from tweise October 16, 2018 14:38
Copy link
Contributor

@tweise tweise left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without it, portable pipeline submission that depends on artifact staging (includes saving main session) is broken in 2.8.


@Option(name = "--flink-master-url", usage = "Flink master url to submit job.")
private String flinkMasterUrl = "[auto]";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why these changes to the modifiers?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing. They are package-local now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few lines further :) There is no need to have a getter, especially not after the visibility of that was changed, too. Somehow it feels as couple recent changes in this area could have benefitted from more thorough review? (BTW I don't expect this to be addressed as part of this PR, but in master.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Considering that the code was more or less untested, I think we are much better off.

We can change the getter back to public.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really a minor issue considering that FlinkJobServerDriver is not user-facing at all.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would probably remove it, but as it stands it is redundant.

The host default change that broke pipeline submission and this PR reverts was user facing. And yes, test coverage added here puts us into a much better place. Test coverage is always preferred, but looking at the PRs a bit more closely sometimes also helps.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we should be careful. That said, all PRs were reviewed and mistakes do happen.

It helps to cut off the release branch a bit earlier so there is some time for testing.

@@ -169,11 +170,6 @@ public void run() {
}
}

public String start() throws IOException {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not be removed (I think we need to adjust this cherry-pick as it introduced other issues in master).

Copy link
Contributor

@tweise tweise left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments.

This adds a few test cases for FlinkJobServerDriver. It also changes the default
host from empty host to `localhost`.
@mxm
Copy link
Contributor Author

mxm commented Oct 16, 2018

@tweise Should be good to go now.

@tweise
Copy link
Contributor

tweise commented Oct 16, 2018

Run Java PreCommit

@tweise
Copy link
Contributor

tweise commented Oct 16, 2018

@aaltay note that we are waiting to merge this PR - it is blocked by unrelated Java pre-commit issues. This PR needs to go into the release.

@tweise
Copy link
Contributor

tweise commented Oct 16, 2018

Run Java PreCommit

1 similar comment
@tweise
Copy link
Contributor

tweise commented Oct 17, 2018

Run Java PreCommit

@tweise tweise merged commit ab28d4a into apache:release-2.8.0 Oct 17, 2018
pl04351820 pushed a commit to pl04351820/beam that referenced this pull request Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants