Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Override RDD count #526

Closed
costin opened this issue Jul 30, 2015 · 1 comment
Closed

Override RDD count #526

costin opened this issue Jul 30, 2015 · 1 comment
Labels
enhancement :Spark v2.1.1 v2.2.0-m1

Comments

@costin
Copy link
Member

costin commented Jul 30, 2015

Currently, invoking RDD count requires all the data to be read from Elastic only to be counted.
This could be improved through a simple count call which would return immediately.
costin added a commit that referenced this issue Aug 21, 2015
@costin
Override RDD count with a fast ES call
eb371b7 
This avoids having to instantiate all the docs
and it does take into account the associated query

relates #526
costin added a commit that referenced this issue Aug 21, 2015
@costin
Override RDD count with a fast ES call
b667000 
This avoids having to instantiate all the docs
and it does take into account the associated query

relates #526

(cherry picked from commit eb371b7)
@costin
Copy link
Member Author

costin commented Aug 27, 2015

Fixed in master and 2.x

@costin costin closed this as completed Aug 27, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement :Spark v2.1.1 v2.2.0-m1
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@costin