Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dynamic concurrent writes #106

Closed
costin opened this issue Nov 2, 2013 · 1 comment
Closed

dynamic concurrent writes #106

costin opened this issue Nov 2, 2013 · 1 comment
Labels
:Cascading feature :Hive :MR :Pig v1.3.0.M2

Comments

@costin
Copy link
Member

costin commented Nov 2, 2013

Just like reading, writing should be dynamic (based on the number of available shards) and parallel to improve performance.
@costin costin mentioned this issue Nov 2, 2013
Closed
costin added a commit that referenced this issue Nov 2, 2013
@costin
add concurrent writes to EsRecordWriter
3495638 
relates #106
costin added a commit that referenced this issue Nov 2, 2013
@costin
enhance writer to wait for index creation
369547b 
relates to #106
costin added a commit that referenced this issue Nov 2, 2013
@costin
invert order of Shard comparison
25969a8 
relates to #106
costin added a commit that referenced this issue Nov 2, 2013
@costin
increase splits in tests even when running on local Hadoop
d71c42d 
relates #106
costin added a commit that referenced this issue Nov 2, 2013
@costin
minor adjustment of Pig test
b2c5090 
relates to #106
costin added a commit that referenced this issue Nov 2, 2013
@costin
update Hive to use concurrent writes
6c4518b 
relates to #106
@costin
Copy link
Member Author

costin commented Nov 2, 2013

pushed to master and fixed.

@costin costin closed this as completed Nov 2, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
:Cascading feature :Hive :MR :Pig v1.3.0.M2
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@costin