Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a blocking variant of close() method to BulkProcessor #4158

Closed
matt-preston opened this Issue Nov 12, 2013 · 3 comments

Comments

Projects
None yet
5 participants
@matt-preston
Copy link
Contributor

matt-preston commented Nov 12, 2013

Using BulkProcessor I often want to block when calling BulkProcessor.close() until all BulkRequests have completed.

Currently this isn't possible. If you use concurrentRequests > 0, calling close() will execute a new bulk request asynchronously then return immediately.

It's possible to write a wrapper around BulkProcessor to implement this behaviour, but it's a bit tricky as you have to understand the Exception handling quirks of BulkProcessor and be aware that every call to listener.beforeBulk() will not guarantee a corresponding call to listener.afterBulk().

It would be nice to have this behaviour implemented as part of the core library. I suggest using this method name:

boolean awaitClose(long timeout, TimeUnit unit) throws InterruptedException;

@kimchy

This comment has been minimized.

Copy link
Member

kimchy commented Nov 13, 2013

Sounds good to me!, would you be interested at working on it?

@matt-preston

This comment has been minimized.

Copy link
Contributor Author

matt-preston commented Nov 13, 2013

Absolutely, I have something working locally, so I'll finish it up and submit a pull request.

matt-preston added a commit to matt-preston/elasticsearch that referenced this issue Nov 15, 2013

Implementation of blocking close method
Blocks until all bulk requests have completed.  This fixes elastic#4158

matt-preston added a commit to matt-preston/elasticsearch that referenced this issue Jan 5, 2014

@javanna javanna self-assigned this Jun 13, 2014

matt-preston added a commit to matt-preston/elasticsearch that referenced this issue Jun 22, 2014

Implementation of blocking close method
Blocks until all bulk requests have completed.  This fixes elastic#4158

@javanna javanna removed the adoptme label Jul 11, 2014

@clintongormley clintongormley removed the adoptme label Jul 11, 2014

@s1monw s1monw added v1.4.0 and removed v1.3.0 labels Jul 14, 2014

@s1monw s1monw assigned s1monw and unassigned javanna Jul 17, 2014

@s1monw s1monw removed the review label Jul 17, 2014

@s1monw

This comment has been minimized.

Copy link
Contributor

s1monw commented Jul 17, 2014

Looks good.. I will pull that in soon

s1monw added a commit that referenced this issue Jul 17, 2014

[Bulk] Add blocking close method to BulkProcessor
Blocks until all bulk requests have completed.

Closes #4158
Closes #6314

@s1monw s1monw closed this in 42b71a0 Jul 17, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.