Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: support memory quota #5524

Merged
merged 2 commits into from Sep 25, 2019

Conversation

@BusyJay
Copy link
Contributor

commented Sep 24, 2019

What have you changed?

  • support memory quota
  • fix grpc coredump issue

What is the type of the changes?

  • New features

How is the PR tested?

integration tests

Does this PR affect documentation (docs) or should it be mentioned in the release notes?

Yes.

Does this PR affect tidb-ansible?

Yes.

- support memory quota
- fix grpc coredump issue

Signed-off-by: Jay Lee <busyjaylee@gmail.com>

- Limit the memory size can be used by gRPC
- Default: unlimited. gRPC usually works well to reclaim memory by itself.
- Limit the memory in case OOM is observed. Note that limit the usage can lead to potential stall.

This comment has been minimized.

Copy link
@hunterlxt

hunterlxt Sep 24, 2019

Contributor

what is potential stall?

This comment has been minimized.

Copy link
@BusyJay

BusyJay Sep 24, 2019

Author Contributor

Allocation stall. It will not allocate until there are free slots.

This comment has been minimized.

Copy link
@hunterlxt

hunterlxt Sep 24, 2019

Contributor

If there is no free memory, the connection(Channel) is refused directly. Maybe it should be explained.

## Limit the memory size can be used by gRPC. Default is unlimited.
## gRPC usually works well to reclaim memory by itself. Limit the memory in case OOM
## is observed. Note that limit the usage can lead to potential stall.
# grpc-memory-pool-quota = "32G"

This comment has been minimized.

Copy link
@hunterlxt

hunterlxt Sep 24, 2019

Contributor

grpc-memory-pool-quota represents the memory usage for each connect. So 32G will it be too large?

This comment has been minimized.

Copy link
@BusyJay

BusyJay Sep 24, 2019

Author Contributor

I think when it's attached to the server, it should be a limit to the memory server uses, not just a single connection. Besides, 32G is just a hint for unlimited.

This comment has been minimized.

Copy link
@hunterlxt

hunterlxt Sep 24, 2019

Contributor

Fine

@@ -63,6 +63,7 @@ pub struct Config {
pub grpc_compression_type: GrpcCompressionType,
pub grpc_concurrency: usize,
pub grpc_concurrent_stream: i32,
pub grpc_memory_pool_quota: ReadableSize,

This comment has been minimized.

Copy link
@hunterlxt

hunterlxt Sep 24, 2019

Contributor

why not just use usize just like grpc_concurrency

This comment has been minimized.

Copy link
@BusyJay

BusyJay Sep 24, 2019

Author Contributor

Because it has unit. Values like "32GiB" or "1024MiB" should be valid configuration.

This comment has been minimized.

Copy link
@hunterlxt

hunterlxt Sep 24, 2019

Contributor

Good Point

@BusyJay BusyJay changed the title *: update grpc *: support memory quota Sep 24, 2019
Copy link
Member

left a comment

LGTM

@sre-bot

This comment has been minimized.

Copy link
Collaborator

commented Sep 25, 2019

/run-all-tests

@sre-bot sre-bot merged commit 0132968 into tikv:release-3.0 Sep 25, 2019
6 checks passed
6 checks passed
DCO All commits are signed off!
Details
idc-jenkins-ci-tikv/integration-common-test Jenkins job succeeded.
Details
idc-jenkins-ci-tikv/integration-compatibility-test Jenkins job succeeded.
Details
idc-jenkins-ci-tikv/integration-cop-push-down-test Jenkins job succeeded.
Details
idc-jenkins-ci-tikv/integration-ddl-test Jenkins job succeeded.
Details
idc-jenkins-ci/test Jenkins job succeeded.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.