Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: query requests dedup #1100

Merged
merged 10 commits into from
Jul 28, 2023
Merged

feat: query requests dedup #1100

merged 10 commits into from
Jul 28, 2023

Conversation

baojinri
Copy link
Contributor

Rationale

In order to reduce the execution of the same query requests，then reduce memory consumption when querying.

Detailed Changes

  • Use hashmap to dedup query requests.
  • Only the first query request will be execute, and other same query requests will wait for the execution result of the first request.

Test Plan

Exsiting tests

@baojinri baojinri marked this pull request as draft July 24, 2023 08:23
@baojinri baojinri changed the title feat: query request dedup feat: query requests dedup Jul 24, 2023
@baojinri baojinri marked this pull request as ready for review July 25, 2023 07:36
ShiKaiWi
ShiKaiWi reviewed Jul 25, 2023
View reviewed changes
server/src/grpc/remote_engine_service/mod.rs Outdated Show resolved Hide resolved
server/src/grpc/remote_engine_service/mod.rs Outdated Show resolved Hide resolved
server/src/grpc/remote_engine_service/mod.rs Outdated Show resolved Hide resolved
server/src/grpc/remote_engine_service/mod.rs Show resolved Hide resolved
server/src/grpc/remote_engine_service/dedup_request.rs Outdated Show resolved Hide resolved
server/src/grpc/remote_engine_service/dedup_request.rs Outdated Show resolved Hide resolved
server/src/grpc/remote_engine_service/dedup_request.rs Outdated Show resolved Hide resolved
server/src/grpc/remote_engine_service/dedup_request.rs Outdated Show resolved Hide resolved
server/src/grpc/remote_engine_service/dedup_request.rs Outdated Show resolved Hide resolved
ShiKaiWi
ShiKaiWi reviewed Jul 27, 2023
View reviewed changes
server/src/grpc/remote_engine_service/dedup_requests.rs Outdated Show resolved Hide resolved
server/src/config.rs Outdated Show resolved Hide resolved
ShiKaiWi
ShiKaiWi reviewed Jul 27, 2023
View reviewed changes
server/src/grpc/remote_engine_service/mod.rs Outdated Show resolved Hide resolved
server/src/grpc/remote_engine_service/mod.rs Outdated Show resolved Hide resolved
server/src/grpc/remote_engine_service/mod.rs Outdated Show resolved Hide resolved
ShiKaiWi
ShiKaiWi reviewed Jul 28, 2023
View reviewed changes
server/src/dedup_requests.rs Outdated Show resolved Hide resolved
server/src/config.rs Outdated Show resolved Hide resolved
ShiKaiWi
ShiKaiWi approved these changes Jul 28, 2023
View reviewed changes
Copy link
Member

@ShiKaiWi ShiKaiWi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ShiKaiWi ShiKaiWi merged commit 1607d1d into apache:main Jul 28, 2023
6 checks passed
@baojinri baojinri deleted the feat-query-request-dedup branch December 5, 2023 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Rachelint Rachelint Rachelint left review comments

@tanruixiang tanruixiang tanruixiang left review comments

@ShiKaiWi ShiKaiWi ShiKaiWi approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@baojinri @ShiKaiWi @Rachelint @tanruixiang