log restore: filter backupmeta file by ts to speed up pitr (#61347) by ti-chi-bot · Pull Request #62115 · pingcap/tidb

ti-chi-bot · 2025-07-01T07:52:32Z

This is an automated cherry-pick of #61347

What problem does this PR solve?

Issue Number: ref #61318

Problem Summary:

As described in the issue.

What changed and how does it work?

Improved timestamp-based filtering logic for pitr .
Ensured compatibility with previous and future versions of TiKV by gracefully handling unexpected or legacy formats.

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Support filtering log backup meta files by timestamp to improve PITR performance.

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2025-07-01T07:52:35Z

@3pointer This PR has conflicts, I have hold it.
Please resolve them or ask others to resolve them, then comment /unhold to remove the hold label.

codecov · 2025-07-09T08:12:19Z

Codecov Report

Attention: Patch coverage is 76.19048% with 25 lines in your changes missing coverage. Please review.

Please upload report for BASE (release-8.5@4caea03). Learn more about missing BASE report.

Additional details and impacted files

@@               Coverage Diff                @@
##             release-8.5     #62115   +/-   ##
================================================
  Coverage               ?   57.7814%           
================================================
  Files                  ?       1777           
  Lines                  ?     630817           
  Branches               ?          0           
================================================
  Hits                   ?     364495           
  Misses                 ?     241698           
  Partials               ?      24624

Flag	Coverage Δ
integration	`38.3349% <52.3809%> (?)`
unit	`72.7889% <76.1904%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`52.9278% <0.0000%> (?)`
parser	`∅ <0.0000%> (?)`
br	`63.8007% <0.0000%> (?)`

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

ti-chi-bot · 2025-07-10T05:46:30Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BornChanger, Leavrth

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~br/OWNERS~~ [BornChanger,Leavrth]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2025-07-10T05:46:33Z

[LGTM Timeline notifier]

Timeline:

2025-07-09 08:33:25.642079387 +0000 UTC m=+2075058.365258369: ☑️ agreed by BornChanger.
2025-07-10 05:46:33.009892122 +0000 UTC m=+2151445.733071101: ☑️ agreed by Leavrth.

3pointer · 2025-07-10T06:08:34Z

/test unit-test

tiprow · 2025-07-10T06:08:51Z

@3pointer: No presubmit jobs available for pingcap/tidb@release-8.5

Details

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

3pointer · 2025-07-10T09:47:22Z

/unhold

3pointer · 2025-07-10T12:10:01Z

/test unit-test

tiprow · 2025-07-10T12:10:17Z

@3pointer: No presubmit jobs available for pingcap/tidb@release-8.5

Details

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

3pointer · 2025-07-10T12:47:05Z

/test unit-test

FAIL: //pkg/server/handler/tests:tests_test (shard 10 of 39) (see /home/jenkins/.tidb/tmp/b02793c799fc405e3b616c63a86b7eb2/execroot/main/bazel-out/k8-fastbuild/testlogs/pkg/server/handler/tests/tests_test/shard_10_of_39/test_attempts/attempt_1.log)

ti-chi-bot · 2025-07-10T12:47:09Z

@3pointer: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test build

/test check-dev

/test check-dev2

/test mysql-test

/test pull-br-integration-test

/test pull-unit-test-ddlv1

/test unit-test

The following commands are available to trigger optional jobs:

/test pull-common-test

/test pull-e2e-test

/test pull-integration-common-test

/test pull-integration-copr-test

/test pull-integration-ddl-test

/test pull-integration-e2e-test

/test pull-integration-jdbc-test

/test pull-integration-mysql-test

/test pull-integration-nodejs-test

/test pull-integration-python-orm-test

/test pull-integration-tidb-tools-test

/test pull-lightning-integration-test

/test pull-mysql-client-test

/test pull-sqllogic-test

/test pull-tiflash-test

Use /test all to run the following jobs that were automatically triggered:

pingcap/tidb/release-8.5/pull_br_integration_test

pingcap/tidb/release-8.5/pull_build

pingcap/tidb/release-8.5/pull_check

pingcap/tidb/release-8.5/pull_check2

pingcap/tidb/release-8.5/pull_mysql_test

pingcap/tidb/release-8.5/pull_unit_test

Details

In response to this:

/test unit-test

FAIL: //pkg/server/handler/tests:tests_test (shard 10 of 39) (see /home/jenkins/.tidb/tmp/b02793c799fc405e3b616c63a86b7eb2/execroot/main/bazel-out/k8-fastbuild/testlogs/pkg/server/handler/tests/tests_test/shard_10_of_39/test_attempts/attempt_1.log)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

tiprow · 2025-07-10T12:47:20Z

@3pointer: No presubmit jobs available for pingcap/tidb@release-8.5

Details

In response to this:

/test unit-test

FAIL: //pkg/server/handler/tests:tests_test (shard 10 of 39) (see /home/jenkins/.tidb/tmp/b02793c799fc405e3b616c63a86b7eb2/execroot/main/bazel-out/k8-fastbuild/testlogs/pkg/server/handler/tests/tests_test/shard_10_of_39/test_attempts/attempt_1.log)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

tiprow · 2025-07-10T12:47:24Z

@ti-chi-bot[bot]: No presubmit jobs available for pingcap/tidb@release-8.5

Details

In response to this:

@3pointer: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:
/test build
/test check-dev
/test check-dev2
/test mysql-test
/test pull-br-integration-test
/test pull-unit-test-ddlv1
/test unit-test
The following commands are available to trigger optional jobs:
/test pull-common-test
/test pull-e2e-test
/test pull-integration-common-test
/test pull-integration-copr-test
/test pull-integration-ddl-test
/test pull-integration-e2e-test
/test pull-integration-jdbc-test
/test pull-integration-mysql-test
/test pull-integration-nodejs-test
/test pull-integration-python-orm-test
/test pull-integration-tidb-tools-test
/test pull-lightning-integration-test
/test pull-mysql-client-test
/test pull-sqllogic-test
/test pull-tiflash-test
Use /test all to run the following jobs that were automatically triggered:
pingcap/tidb/release-8.5/pull_br_integration_test
pingcap/tidb/release-8.5/pull_build
pingcap/tidb/release-8.5/pull_check
pingcap/tidb/release-8.5/pull_check2
pingcap/tidb/release-8.5/pull_mysql_test
pingcap/tidb/release-8.5/pull_unit_test
In response to this:

/test unit-test

FAIL: //pkg/server/handler/tests:tests_test (shard 10 of 39) (see /home/jenkins/.tidb/tmp/b02793c799fc405e3b616c63a86b7eb2/execroot/main/bazel-out/k8-fastbuild/testlogs/pkg/server/handler/tests/tests_test/shard_10_of_39/test_attempts/attempt_1.log)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

3pointer · 2025-07-10T12:55:34Z

/retest

3pointer · 2025-07-10T13:25:54Z

/test unit-test

tiprow · 2025-07-10T13:26:10Z

@3pointer: No presubmit jobs available for pingcap/tidb@release-8.5

Details

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

…1347) (pingcap#62115) (pingcap#264)

…1347) (pingcap#62115) X-Original-Commit: b9e2ec9

This is an automated cherry-pick of pingcap#61347

bd9b3e2

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot assigned 3pointer Jul 1, 2025

ti-chi-bot mentioned this pull request Jul 1, 2025

log restore: filter backupmeta file by ts to speed up pitr #61347

Merged

13 tasks

ti-chi-bot Bot added the do-not-merge/cherry-pick-not-approved label Jul 1, 2025

ti-chi-bot Bot added cherry-pick-approved Cherry pick PR approved by release team. and removed do-not-merge/cherry-pick-not-approved labels Jul 8, 2025

3pointer added 2 commits July 9, 2025 15:27

Merge branch 'release-8.5' into cherry-pick-61347-to-release-8.5

d176c6b

resolve conflict

f39217a

BornChanger approved these changes Jul 9, 2025

View reviewed changes

ti-chi-bot Bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 9, 2025

fix bazel

5181d78

Leavrth approved these changes Jul 10, 2025

View reviewed changes

ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 10, 2025

ti-chi-bot Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 10, 2025

ti-chi-bot Bot merged commit c780f4e into pingcap:release-8.5 Jul 10, 2025
23 checks passed

AvaisP added a commit to AvaisP/tidb that referenced this pull request Mar 2, 2026

log restore: filter backupmeta file by ts to speed up pitr (pingcap#6…

6bd02fd

…1347) (pingcap#62115) (pingcap#264)

AvaisP added a commit to AvaisP/tidb that referenced this pull request Mar 2, 2026

log restore: filter backupmeta file by ts to speed up pitr (pingcap#6…

533dfdf

…1347) (pingcap#62115) (pingcap#264)

AvaisP added a commit to AvaisP/tidb that referenced this pull request Mar 4, 2026

log restore: filter backupmeta file by ts to speed up pitr (pingcap#6…

0566c82

…1347) (pingcap#62115) (pingcap#264)

ClamChowderTiDB added a commit to ClamChowderTiDB/tidb that referenced this pull request Mar 31, 2026

log restore: filter backupmeta file by ts to speed up pitr (pingcap#6…

79d75da

…1347) (pingcap#62115) X-Original-Commit: b9e2ec9

Conversation

ti-chi-bot commented Jul 1, 2025

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

Uh oh!

ti-chi-bot commented Jul 1, 2025

Uh oh!

codecov Bot commented Jul 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ti-chi-bot Bot commented Jul 10, 2025

Uh oh!

ti-chi-bot Bot commented Jul 10, 2025

[LGTM Timeline notifier]

Uh oh!

3pointer commented Jul 10, 2025

Uh oh!

tiprow Bot commented Jul 10, 2025

Uh oh!

3pointer commented Jul 10, 2025

Uh oh!

3pointer commented Jul 10, 2025

Uh oh!

tiprow Bot commented Jul 10, 2025

Uh oh!

3pointer commented Jul 10, 2025

Uh oh!

ti-chi-bot Bot commented Jul 10, 2025

Uh oh!

tiprow Bot commented Jul 10, 2025

Uh oh!

tiprow Bot commented Jul 10, 2025

Uh oh!

3pointer commented Jul 10, 2025

Uh oh!

3pointer commented Jul 10, 2025

Uh oh!

tiprow Bot commented Jul 10, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov Bot commented Jul 9, 2025 •

edited

Loading