Skip to content

[Feature](ES): Es support sort node pushdown limit and date type#16583

Closed
stalary wants to merge 8 commits intoapache:masterfrom
stalary:feature/es_pushdown_order_limit
Closed

[Feature](ES): Es support sort node pushdown limit and date type#16583
stalary wants to merge 8 commits intoapache:masterfrom
stalary:feature/es_pushdown_order_limit

Conversation

@stalary
Copy link
Contributor

@stalary stalary commented Feb 10, 2023

Proposed changes

Issue Number: close #xxx

Problem summary

  1. Es support sort node pushdown limit, resolve limit speed too slow when use order by.
  2. fix error mapping doris type when es use date type.

Checklist(Required)

  1. Does it affect the original behavior:
    • Yes
    • No
    • I don't know
  2. Has unit tests been added:
    • Yes
    • No
    • No Need
  3. Has document been added or modified:
    • Yes
    • No
    • No Need
  4. Does it need to update dependencies:
    • Yes
    • No
  5. Are there any changes that cannot be rolled back:
    • Yes (If Yes, please explain WHY)
    • No

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actions github-actions bot added the area/planner Issues or PRs related to the query planner label Feb 10, 2023
@stalary stalary changed the title (Feature)(ES): Es support sort node pushdown limit [Feature](ES): Es support sort node pushdown limit Feb 10, 2023
@stalary stalary closed this Feb 10, 2023
@stalary stalary reopened this Feb 10, 2023
@stalary stalary changed the title [Feature](ES): Es support sort node pushdown limit [Feature](ES): Es support sort node pushdown limit and date type Feb 10, 2023
@stalary stalary force-pushed the feature/es_pushdown_order_limit branch from 8308f2c to 587630d Compare February 10, 2023 14:40
@hello-stephen
Copy link
Contributor

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 35.88 seconds
stream load tsv: 478 seconds loaded 74807831229 Bytes, about 149 MB/s
stream load json: 41 seconds loaded 2358488459 Bytes, about 54 MB/s
stream load orc: 68 seconds loaded 1101869774 Bytes, about 15 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230210210634_clickbench_pr_94301.html

@stalary stalary closed this Feb 13, 2023
@stalary stalary deleted the feature/es_pushdown_order_limit branch February 16, 2023 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/planner Issues or PRs related to the query planner kind/test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants