Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document update with logic/script. #196

Closed
jwinder opened this Issue Apr 23, 2014 · 1 comment

Comments

Projects
None yet
2 participants
@jwinder
Copy link

commented Apr 23, 2014

Conversation in IRC:

https://gist.github.com/jwinder/23f346da2fcd87bfc0e4

I have a situation where I would like to construct lists of values in array objects such that my scalding job will be able to output different items that can be collected to the same document in elasticsearch but having elasticsearch append the values to the array object in the correct document instead of overwrite the array -- ending with a single element array for each document.

I have found that accumulating entire lists in memory within the jobs can lead to memory problems.

@costin costin added hive labels Apr 25, 2014

costin added a commit that referenced this issue Apr 25, 2014

@costin

This comment has been minimized.

Copy link
Member

commented Apr 25, 2014

@jwinder Hi,
I've pushed this feature into master - it's not fully tested however it is functionality complete. Until the docs get updated, you can check out the MRSuite tests. You can use as operation, between upsert and update and have a number of new params which you can find here

costin added a commit that referenced this issue Apr 29, 2014

@costin costin closed this May 8, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.