Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore data locality preference for hadoop and spark #814

Closed
jimczi opened this issue Jul 28, 2016 · 0 comments
Closed

Restore data locality preference for hadoop and spark #814

jimczi opened this issue Jul 28, 2016 · 0 comments

Comments

@jimczi
Copy link
Contributor

jimczi commented Jul 28, 2016

The data locality preference has been dropped in #812. We should restore the functionality but instead of setting one node per partition we should let hadoop/spark choose among the nodes that host the index/shard targeted by the partition. Since it's only a preference, having multiple hosts that could serve the query is beneficial in terms of execution. The list of hosts should be shuffled in order to spread the execution of multiple partitions that target the same index/shard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants