Skip to content

Conversation

@ShiKaiWi
Copy link
Member

@ShiKaiWi ShiKaiWi commented Aug 10, 2023

Rationale

According to the cpu profile results, it costs too much cpu resource when encode/decode the payloads for the remote engine write service.

Detailed Changes

  • Impose a new protocol for communication between remote engine client and service
  • Avoid attaching the schema information in the payload
  • Do the encoding work outside the io thread pool

Test Plan

Proved 60% traffic drop and 25% cpu performance improvement in one of our prod environment.

@ShiKaiWi ShiKaiWi force-pushed the improve-remote-engine-service branch from 83a891c to 35d1d81 Compare August 14, 2023 01:54
Copy link
Contributor

@jiacai2050 jiacai2050 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ShiKaiWi ShiKaiWi merged commit 4185b66 into apache:main Aug 14, 2023
@ShiKaiWi ShiKaiWi deleted the improve-remote-engine-service branch August 14, 2023 04:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants