Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-6612] Improve java SDK performance by using a LinkedBlockingQueue in QueueingBeamFnDataClient #7895

Merged

Conversation

ajamato
Copy link

@ajamato ajamato commented Feb 19, 2019

[BEAM-6612] Improve java SDK performance by using a LinkedBlockingQueue in QueueingBeamFnDataClient

Now a small queue size is used instead of a synchronous queue which only held one element.


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Choose reviewer(s) and mention them in a comment (R: @username).
  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

Post-Commit Tests Status (on master branch)

Lang SDK Apex Dataflow Flink Gearpump Samza Spark
Go Build Status --- --- --- --- --- ---
Java Build Status Build Status Build Status Build Status
Build Status
Build Status
Build Status Build Status Build Status
Python Build Status --- Build Status
Build Status
Build Status --- --- ---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

@ajamato ajamato force-pushed the buffering_queue_linked_blocking_queue branch from cc51416 to 53b76b6 Compare February 27, 2019 17:33
@ajamato ajamato changed the title [Do not merge] Buffering queue linked blocking queue [Do not merge] [BEAM-6612] Improve java SDK performance by using a LinkedBlockingQueue Feb 27, 2019
@ajamato ajamato force-pushed the buffering_queue_linked_blocking_queue branch 2 times, most recently from 12a8dcc to 269952b Compare February 28, 2019 18:54
@ajamato ajamato force-pushed the buffering_queue_linked_blocking_queue branch from 269952b to 5ccdd32 Compare February 28, 2019 19:00
@ajamato
Copy link
Author

ajamato commented Feb 28, 2019

@mdvorsky @kennknowles

@ajamato
Copy link
Author

ajamato commented Mar 1, 2019

Run Java PreCommit

@ajamato ajamato changed the title [Do not merge] [BEAM-6612] Improve java SDK performance by using a LinkedBlockingQueue [BEAM-6612] Improve java SDK performance by using a LinkedBlockingQueue Mar 1, 2019
@ajamato
Copy link
Author

ajamato commented Mar 1, 2019

R: @mdvorsky
R: @kennknowles

@ajamato ajamato changed the title [BEAM-6612] Improve java SDK performance by using a LinkedBlockingQueue [BEAM-6612] Improve java SDK performance by using a LinkedBlockingQueue in QueueingBeamFnDataClient Mar 1, 2019
@kennknowles kennknowles self-requested a review March 2, 2019 02:14
@@ -65,6 +65,8 @@ public void complete() {

@Override
public void fail(Throwable t) {
future.completeExceptionally(t);
// Use obtrudeException instead of CompleteExceptionally, forcing any future calls to .get()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I did not realize that standard future behavior required this other function :-/

@kennknowles kennknowles merged commit 1107fc1 into apache:master Mar 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants