tikv_alloc: Add per thread memory usage

[Connor1996]

What is changed and how it works?

Issue Number: Ref #15927

What's Changed:

Add per thread memory usage

Jemalloc has mapped memory stats per arena. We can bind each thread with a dedicated arena, then we have per thread memory usage.

The total memory ~== block cache size + total of per thread footprint

Related changes

• PR to update pingcap/docs/pingcap/docs-cn:
• Need to cherry-pick to the release branch

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Release note

Add per thread memory usage

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• glorv
• overvenus

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[overvenus]

Does it mean memory arena can not be shared between threads?
Will it aggravate memory fragment or even OOM?
Can it tell how much memory is held (but not allocated) by a thread?

[Connor1996]

• By default, the number of arena is four times of CPU cores. So currently, it's already nearly one arena per one thread. So I don't think it would aggravate memory fragment in TiKV case.

• What you mean by "held" is holding the memory allocated by other thread? Can't as allocator doesn't have the knowledge that the memory ownership is moved to another thread. At least, when it's freed in other threads, the memory usage accounted in the original allocated thread would be decreased.

[overvenus]

Should we consider implementing a configuration switch to disable this feature, as a precautionary measure in case it triggers unforeseen issues?

[tonyxuqqi]

According to the document, " There is a small fixed per-arena overhead, and additionally, arenas manage memory completely independently of each other, which means a small fixed increase in overall memory fragmentation. These overheads are not generally an issue, given the number of arenas normally used. Note that using substantially more arenas than the default is not likely to improve performance, mainly due to reduced cache performance. However, it may make sense to reduce the number of arenas if an application does not make much use of the allocation functions."

To make sure the overhead is small, can you run some read write mixed workload to compare the memory usage with and without this feature? @Connor1996

[Connor1996]

There are four rounds of sysbench tests, read_only, insert, write_only, read_write case respectively. And there is no noticeable difference in memory usage

[Connor1996]

/test

[ti-chi-bot]

@Connor1996: The /test command needs one or more targets.
The following commands are available to trigger required jobs:

• /test pull-unit-test

Use /test all to run all jobs.

In response to this:

/test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Connor1996]

/merge

[ti-chi-bot]

@Connor1996: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: 7a3df1d

[glorv]

@Connor1996 you need to address the DCO first

[Connor1996]

/test

[ti-chi-bot]

@Connor1996: The /test command needs one or more targets.
The following commands are available to trigger required jobs:

• /test pull-unit-test

Use /test all to run all jobs.

In response to this:

/test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Connor1996]

/test pull-unit-test

[Connor1996]

/test pull-unit-test

[glorv]

--- TRY 3 STDERR:        tikv config::tests::test_config_template_no_superfluous_keys ---

thread 'config::tests::test_config_template_no_superfluous_keys' panicked at src/config/mod.rs:6878:10:

called `Result::unwrap()` on an `Err` value: Error { inner: ErrorInner { kind: Wanted { expected: "newline", found: "a comma" }, line: Some(99), col: 37, at: Some(4428), message: "", key: [] } }

stack backtrace:

   0: rust_begin_unwind

             at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library/std/src/panicking.rs:645:5

   1: core::panicking::panic_fmt

             at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library/core/src/panicking.rs:72:14

   2: core::result::unwrap_failed

             at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library/core/src/result.rs:1649:5

   3: core::result::Result<T,E>::unwrap

   4: tikv::config::tests::test_config_template_no_superfluous_keys

   5: tikv::config::tests::test_config_template_no_superfluous_keys::{{closure}}

   6: core::ops::function::FnOnce::call_once

   7: core::ops::function::FnOnce::call_once

             at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library/core/src/ops/function.rs:250:5

note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

@Connor1996 Seems on test case need to be fixed

[Connor1996]

/merge

[ti-chi-bot]

@Connor1996: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: c4fdfa9

[Connor1996]

/cherry-pick release-8.1

[ti-chi-bot]

@Connor1996: new pull request created to branch release-8.1: #17005.

In response to this:

/cherry-pick release-8.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.