Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eliminate object saving when performing bulk/buffer updates #16

Closed
costin opened this issue Apr 3, 2013 · 4 comments
Closed

eliminate object saving when performing bulk/buffer updates #16

costin opened this issue Apr 3, 2013 · 4 comments

Comments

@costin
Copy link
Member

costin commented Apr 3, 2013

Currently to perform efficient writes, the data is saved before being passed to ES. As Hadoop (and various libraries) perform object pooling, each entry needs to be copied otherwise the data is lost.
This causes significant memory overhead which can be alleviated by serializing early ( #3 )

@costin
Copy link
Member Author

costin commented Apr 8, 2013

@CodeMomentum Could you expand a bit on the workflow and what do you mean by "other atlernatives"? Is this somehow related to bulk updates? If not, why not raise a separate issue?

@lorthos
Copy link

lorthos commented Apr 8, 2013

@costin You are right, I created a separate issue, sorry for the confusion

@costin
Copy link
Member Author

costin commented Apr 8, 2013

@CodeMomentum no worries.

@costin
Copy link
Member Author

costin commented Apr 18, 2013

Fixed in master. The object content is saved early on as byte[] so the objects themselves can be safely reused.

@costin costin closed this as completed Apr 18, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants