Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(interactive): Reuse compiled native lib for same physical plan #3301

Merged
merged 9 commits into from
Oct 25, 2023

Conversation

zhanglei1949
Copy link
Collaborator

@zhanglei1949 zhanglei1949 commented Oct 19, 2023

Add a server-side query cache on hqps engine.

  • Add plan_id field in PhysicalPlan.
  • Maintain the mapping between plan_id between compiled dynamic lib path.
  • Avoid unnecessary recompilation for same physical plan.
  • Each shard in hqps engine will have exactly one actor handling codegen request, which is actually delegated to hiactor::thread_resource_pool. And the thread will not be blocked.

@zhanglei1949 zhanglei1949 marked this pull request as draft October 19, 2023 07:27
@zhanglei1949 zhanglei1949 marked this pull request as ready for review October 21, 2023 07:45
@codecov-commenter
Copy link

codecov-commenter commented Oct 21, 2023

Codecov Report

Merging #3301 (60e594e) into main (7a7bd4e) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #3301   +/-   ##
=======================================
  Coverage   41.39%   41.39%           
=======================================
  Files         107      107           
  Lines       11203    11203           
=======================================
  Hits         4637     4637           
  Misses       6566     6566           

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7a7bd4e...60e594e. Read the comment docs.

@zhanglei1949 zhanglei1949 changed the title feat(Interactive): Reuse compiled native lib for same physical plan feat(interactive): Reuse compiled native lib for same physical plan Oct 24, 2023
@luoxiaojian luoxiaojian requested a review from LiSu October 24, 2023 02:54
Copy link
Collaborator

@LiSu LiSu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The actor part LGTM.

@zhanglei1949 zhanglei1949 merged commit 8bb913a into alibaba:main Oct 25, 2023
46 of 48 checks passed
@zhanglei1949 zhanglei1949 deleted the query-cache branch March 14, 2024 02:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants