Skip to content

Reset current_memory_tracker when WNEstablishDisaggTaskHandler finished.#8174

Merged
ti-chi-bot[bot] merged 3 commits intopingcap:masterfrom
JinheLin:fix_mem_tracker_in_EstablishDisaggTask
Oct 9, 2023
Merged

Reset current_memory_tracker when WNEstablishDisaggTaskHandler finished.#8174
ti-chi-bot[bot] merged 3 commits intopingcap:masterfrom
JinheLin:fix_mem_tracker_in_EstablishDisaggTask

Conversation

@JinheLin
Copy link
Copy Markdown
Contributor

@JinheLin JinheLin commented Oct 9, 2023

What problem does this PR solve?

Issue Number: close #8173

Problem Summary:

  1. In WNEstablishDisaggTaskHandler::execute, a ProcessListElement object will be created and sets current_memory_tracker to its memory_tracker member in constructor. And current_memory_tracker will be reset in the destructor of ProcessListElement.

  2. However, objects of ProcessListElement are created in threads of WNEstablishDisaggTaskPool, but destructed in threads of GRPC because it will be destructed with object of WNEstablishDisaggTaskHandler (handler -> dag_context -> process_list_entry).

  3. This result in current_memory_tracker of threads of WNEstablishDisaggTaskPool is not reset after task finished and other tasks may continue to use this memory tracker, although it may have already been destructed.

What is changed and how it works?

  • Reset current_memory_tracker after tasks of WNEstablishDisaggTaskPool finished.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Oct 9, 2023
@JinheLin
Copy link
Copy Markdown
Contributor Author

JinheLin commented Oct 9, 2023

/run-all-tests

{
auto task = std::make_shared<std::packaged_task<void()>>(
[&handler, &request, &response, deadline = grpc_context->deadline()]() {
[db_context = db_context, &task_id, &request, &response, deadline = grpc_context->deadline()]() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
[db_context = db_context, &task_id, &request, &response, deadline = grpc_context->deadline()]() {
[&db_context, &task_id, &request, &response, deadline = grpc_context->deadline()]() {

maybe better?

Copy link
Copy Markdown
Contributor Author

@JinheLin JinheLin Oct 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'db_context' in capture list does not name a variable

Lambda could not capture structured binding in C++17.

Ref: https://stackoverflow.com/questions/46114214/lambda-implicit-capture-fails-with-variable-declared-from-structured-binding

@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Oct 9, 2023
@ti-chi-bot
Copy link
Copy Markdown
Contributor

ti-chi-bot bot commented Oct 9, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: guo-shaoge, JaySon-Huang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [JaySon-Huang,guo-shaoge]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Oct 9, 2023
@ti-chi-bot
Copy link
Copy Markdown
Contributor

ti-chi-bot bot commented Oct 9, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-10-09 08:16:27.899663522 +0000 UTC m=+1040185.486773652: ☑️ agreed by JaySon-Huang.
  • 2023-10-09 08:20:11.770941297 +0000 UTC m=+1040409.358051443: ☑️ agreed by guo-shaoge.

@JinheLin
Copy link
Copy Markdown
Contributor Author

JinheLin commented Oct 9, 2023

/run-all-tests

@ti-chi-bot ti-chi-bot bot merged commit 952c911 into pingcap:master Oct 9, 2023
@JaySon-Huang
Copy link
Copy Markdown
Contributor

/cherry-pick release-7.4

@ti-chi-bot
Copy link
Copy Markdown
Member

@JaySon-Huang: new pull request created to branch release-7.4: #8175.

Details

In response to this:

/cherry-pick release-7.4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot bot pushed a commit that referenced this pull request Oct 10, 2023
guo-shaoge pushed a commit to guo-shaoge/tiflash that referenced this pull request Nov 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm release-note-none Denotes a PR that doesn't merit a release note. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TiFlash process crashes in MemoryTracker

4 participants