ddl: check local file existence before resume checkpoint #53072
Conversation
Signed-off-by: lance6716 <lance6716@gmail.com>
ti-chi-bot
bot
added
release-note-none
size/XL
|
/ok-to-test |
Signed-off-by: lance6716 <lance6716@gmail.com>
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #53072 +/- ##
================================================
+ Coverage 72.0130% 72.2304% +0.2173%
================================================
Files 1499 1508 +9
Lines 431223 438282 +7059
================================================
+ Hits 310537 316573 +6036
- Misses 101378 102014 +636
- Partials 19308 19695 +387
Flags with carried forward coverage won't be shown. Click here to find out more.
|
ti-chi-bot
bot
added
size/XXL
pkg/ddl/ingest/checkpoint.go
Outdated
| localCnt int | ||
| globalCnt int | ||
| minKeyFlushed kv.Key | ||
| minKeyImported kv.Key |
There was a problem hiding this comment.
shouldn't we call it maxKeyImported, it means all keys < it are imported
There was a problem hiding this comment.
ChatGPT still prefer the minXXX
You
How to name a variable which means "all keys less than it are flushed, but the keys larger than it can be flushed or not flushed". And it's calculated from the minimum value of some concurrency worker's progress.
ChatGPT
You might consider naming the variable something descriptive yet succinct, like "minFlushedKey" or "flushThreshold". This would indicate that it represents the minimum key value up to which all keys are flushed, but keys larger than it might or might not be flushed depending on their status. Alternatively, you could name it "minFlushedProgressKey" to be more explicit about its relationship to progress.
Here's a breakdown of each potential name:
minFlushedKey: This name clearly indicates that it represents the minimum key value that has been flushed. It suggests that any keys smaller than this value are flushed, while those larger than it may or may not be flushed.
flushThreshold: This name implies that it's a point or level that determines when flushing occurs. It's like a threshold that, once reached, triggers the flushing of keys. "FlushThreshold" conveys that keys below this threshold are flushed, while those above it may or may not be.
minFlushedProgressKey: This name explicitly links the variable to progress and flushing. It suggests that it represents the minimum key based on the progress made, and keys less than or equal to this value are considered flushed.
Choose the name that best fits the context and naming conventions of your project. The most important thing is that the name is clear and understandable to anyone reading your code.
There was a problem hiding this comment.
i don't think minFlushedKey suggests that any keys smaller than this value are flushed, it only states that it's the min key that have flushed and any key before or after it may or may not flushed
And it's calculated from the minimum value of some concurrency worker's progress.
from this description, we can only got a name like min key of on-going task
There was a problem hiding this comment.
How about rename it to flushedKeyLowWatermark? I can do it in #52993
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: D3Hunter, tangenta The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Signed-off-by: lance6716 <lance6716@gmail.com>
|
/cherry-pick release-8.1 |
|
@lance6716: new pull request created to branch In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
|
/cherry-pick release-7.5 |
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
|
@wjhuang2016: new pull request created to branch In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
|
In response to a cherrypick label: new pull request could not be created: failed to create pull request against pingcap/tidb#release-7.5 from head ti-chi-bot:cherry-pick-53072-to-release-7.5: status code 422 not one of [201], body: {"message":"Validation Failed","errors":[{"resource":"PullRequest","code":"custom","message":"A pull request already exists for ti-chi-bot:cherry-pick-53072-to-release-7.5."}],"documentation_url":"https://docs.github.com/rest/pulls/pulls#create-a-pull-request"} |
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
What problem does this PR solve?
Issue Number: close #53009
Problem Summary:
What changed and how does it work?
FlushModeForceLocalAndCheckDiskQuotabecause it's not usedctxto detect the closing ofCheckpointManagerand let other goroutine communications check the contextCheckpointManagerupdatingfield because only one goroutine callsupdateCheckpointCheck List
Tests
Side effects
Documentation
Release note
Please refer to Release Notes Language Style Guide to write a quality release note.