Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit pushdown for Elasticsearch connector #2781

Merged
merged 1 commit into from
Feb 11, 2020

Conversation

Lewuathe
Copy link
Member

Summary

Implement limit pushdown to Elasticsearch connector

See: #2624

Overview

  • Extend ElasticsearchTableHandle to store limit information
  • Change iterator to stop reading records when it exceeds the limit
  • Override scroll size with the given limit

@martint martint merged commit 98db07b into trinodb:master Feb 11, 2020
@martint martint mentioned this pull request Feb 11, 2020
7 tasks
@martint martint added this to the 330 milestone Feb 11, 2020
@Lewuathe Lewuathe deleted the elasticsearch-apply-limit branch February 11, 2020 23:44
@armsword
Copy link

armsword commented Sep 3, 2020

@Lewuathe Hi, why the second parameter "limitGuaranteed" in LimitApplicationResult is set to false?

@Lewuathe
Copy link
Member Author

Lewuathe commented Sep 3, 2020

@armsword Since the limit is enforced by PageSource, the number of records returned by a PageSource is guaranteed to be less than the limit. But there is no guarantee in total from the Elasticsearch connector.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

3 participants