Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

log-backup: adapt user-defined TLS config #12640

Merged
merged 13 commits into from May 30, 2022
Merged

Conversation

YuJuncen
Copy link
Contributor

@YuJuncen YuJuncen commented May 25, 2022

Signed-off-by: Yu Juncen yujuncen@pingcap.com

What is changed and how it works?

Issue Number: Close #12668

What's Changed:

  • Added a new metastore type LazyEtcd, which supports connecting with TLS.
  • Modify the Endpoint::new, now it receives MetaStore instead of endpoints.
log-backup: adapt user-defined TLS config

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn:
  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
    I have tested it by:
  1. Create a cluster by tiup.
  2. Enable TLS by tiup cluster tls <cluster-name> enable.
  3. Run log backup, there should not be errors or panic.
  • No code

Release note

Fix a bug that caused PiTR cannot work if some of `security.{ca-path, cert-path, key-path}` configured.

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>
@ti-chi-bot
Copy link
Member

ti-chi-bot commented May 25, 2022

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • 3pointer
  • kennytm

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

YuJuncen and others added 4 commits May 25, 2022 14:34
Signed-off-by: Yu Juncen <yujuncen@pingcap.com>
Signed-off-by: Yu Juncen <yujuncen@pingcap.com>
Signed-off-by: Yu Juncen <yujuncen@pingcap.com>
@@ -632,6 +551,7 @@ where
.await
.map_err(|err| Error::from(err).report("failed to get tso from pd"))
.unwrap_or_default();
cm.update_max_ts(pd_tso);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why move this line?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should update this to pd ts before we trying to resolve, so there won't be new async-commit prewrite uses [0, pd_ts] to commit during we resolving.

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>
Signed-off-by: Yu Juncen <yujuncen@pingcap.com>
@YuJuncen YuJuncen requested review from kennytm and 3pointer May 26, 2022 10:53
Copy link
Contributor

@3pointer 3pointer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added the status/LGT1 Status: PR - There is already 1 approval label May 26, 2022
Copy link
Contributor

@kennytm kennytm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest lgtm

components/backup-stream/src/endpoint.rs Outdated Show resolved Hide resolved
@ti-chi-bot ti-chi-bot added status/LGT2 Status: PR - There are already 2 approvals and removed status/LGT1 Status: PR - There is already 1 approval labels May 30, 2022
Signed-off-by: Yu Juncen <yujuncen@pingcap.com>
@kennytm
Copy link
Contributor

kennytm commented May 30, 2022

/merge

@ti-chi-bot
Copy link
Member

@kennytm: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: f2a6cdc

@ti-chi-bot ti-chi-bot added the status/can-merge Status: Can merge to base branch label May 30, 2022
@ti-chi-bot ti-chi-bot merged commit 01ad5fe into tikv:master May 30, 2022
ti-srebot pushed a commit to ti-srebot/tikv that referenced this pull request May 30, 2022
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor

cherry pick to release-6.1 in PR #12699

joccau pushed a commit to joccau/tikv that referenced this pull request Jun 10, 2022
close tikv#12668

log-backup: adapt user-defined TLS config

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

Co-authored-by: kennytm <kennytm@gmail.com>
joccau added a commit to joccau/tikv that referenced this pull request Jun 11, 2022
br: log-backup: adapt user-defined TLS config (tikv#12640)
joccau pushed a commit to joccau/tikv that referenced this pull request Jun 23, 2022
close tikv#12668

log-backup: adapt user-defined TLS config

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

Co-authored-by: kennytm <kennytm@gmail.com>
Signed-off-by: joccau <zak.zhao@pingcap.com>
WangLe1321 added a commit to WangLe1321/tikv that referenced this pull request Jun 23, 2022
commit 900dd03
Merge: 9fda729 8eceb1f
Author: Zak Zhao <57036248+joccau@users.noreply.github.com>
Date:   Thu Jun 23 12:12:48 2022 +0800

    Merge pull request tikv#9 from joccau/fix-limit-fluent

    log-backup: limit concurrent fluent when write kv-files to storage

commit 8eceb1f
Author: joccau <zak.zhao@pingcap.com>
Date:   Thu Jun 23 11:21:20 2022 +0800

    move local variables to const

    Signed-off-by: joccau <zak.zhao@pingcap.com>

commit 4465085
Author: joccau <zak.zhao@pingcap.com>
Date:   Wed Jun 22 16:40:48 2022 +0800

    add unit test

    Signed-off-by: joccau <zak.zhao@pingcap.com>

commit 8ef93a1
Author: joccau <zak.zhao@pingcap.com>
Date:   Wed Jun 22 00:37:46 2022 +0800

    limit the fluent when write kv file, ref issues#12869

    Signed-off-by: joccau <zak.zhao@pingcap.com>

commit 9fda729
Merge: d219d84 acc06c3
Author: Zak Zhao <57036248+joccau@users.noreply.github.com>
Date:   Wed Jun 15 17:28:24 2022 +0800

    Merge pull request tikv#8 from YuJuncen/update-max-ts

    log_backup: update max ts

commit acc06c3
Author: Yu Juncen <yujuncen@pingcap.com>
Date:   Wed Jun 15 16:37:33 2022 +0800

    *: update max ts

    Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

commit d219d84
Merge: baf35ea e6d1dfe
Author: Zak Zhao <57036248+joccau@users.noreply.github.com>
Date:   Wed Jun 15 13:50:32 2022 +0800

    Merge pull request tikv#7 from WangLe1321/feat/br_stream_metric

    br: remove metrics after task stopped

commit e6d1dfe
Author: WangLe1321 <wangle1321@163.com>
Date:   Tue Jun 14 17:55:47 2022 +0800

    br: modify to recieve returned value

commit 9b1c101
Author: WangLe1321 <wangle1321@163.com>
Date:   Tue Jun 14 17:47:20 2022 +0800

    feat: check result after remove metric

commit 2619864
Author: WangLe1321 <wangle1321@163.com>
Date:   Tue Jun 14 15:11:38 2022 +0800

    br: remove metrics when task stopped

commit baf35ea
Merge: b1d493c 4b6ee85
Author: Zak Zhao <57036248+joccau@users.noreply.github.com>
Date:   Mon Jun 13 10:55:23 2022 +0800

    Merge pull request tikv#6 from joccau/error-safe-point

    log-backup: update flush time if succeeding in do_flush

commit 4b6ee85
Author: joccau <zak.zhao@pingcap.com>
Date:   Sat Jun 11 17:38:47 2022 +0800

    update flush time if succeeding in do_flush, ref issue#12802

    Signed-off-by: joccau <zak.zhao@pingcap.com>

commit b1d493c
Merge: 0e445f8 c9b5f06
Author: Zak Zhao <57036248+joccau@users.noreply.github.com>
Date:   Sat Jun 11 13:51:02 2022 +0800

    Merge pull request tikv#5 from joccau/cherry-pick-12770

    log_backup: adapt the enhanced checkpoint model

commit c9b5f06
Author: Yu Juncen <yujuncen@pingcap.com>
Date:   Thu Jun 9 11:58:33 2022 +0800

    fix build && make clippy happy

    Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

commit 7b83eec
Author: Yu Juncen <yujuncen@pingcap.com>
Date:   Mon Jun 6 15:46:26 2022 +0800

    fix build && make clippy happy

    Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

commit 228cf35
Author: Yu Juncen <yujuncen@pingcap.com>
Date:   Mon Jun 6 11:07:46 2022 +0800

    Squashed commit of the following:

    commit dca2e4efec59a47299d56c9469164f12eef978b7
    Author: Yu Juncen <yujuncen@pingcap.com>
    Date:   Thu Jun 2 18:12:13 2022 +0800

        make slash_etc support mvcc

        Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

    commit 0ead359
    Author: Yu Juncen <yujuncen@pingcap.com>
    Date:   Tue May 31 19:13:43 2022 +0800

        *: make

        Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

    commit 751f0fa
    Author: Yu Juncen <yujuncen@pingcap.com>
    Date:   Tue May 31 14:04:33 2022 +0800

        make 2-phase flushing

        Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

    commit a0981dc
    Author: Yu Juncen <yujuncen@pingcap.com>
    Date:   Tue May 31 13:07:16 2022 +0800

        make initial scanning sync with region operator

        Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

    commit 5b091ea
    Author: Yu Juncen <yujuncen@pingcap.com>
    Date:   Mon May 30 13:44:39 2022 +0800

        WIP: register region checkpoint, keep checkpoint until flush

        Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

    commit c0fa5dd
    Author: Yu Juncen <yujuncen@pingcap.com>
    Date:   Sat May 28 20:50:16 2022 +0800

        added wait group

        Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

    commit cf2c4ff
    Author: Yu Juncen <yujuncen@pingcap.com>
    Date:   Sat May 28 19:58:03 2022 +0800

        Squashed commit of the following:

        commit 09e9aa53ffdf320a956409b7b220ff1f68283b47
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Sat May 28 13:01:49 2022 +0800

            basic change

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit 505bcc7c548165a55ac22c5258ee39f0ab7a8937
        Merge: ac005ba42 93fecdc
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Fri May 27 11:59:00 2022 +0800

            Merge branch 'master' of https://github.com/tikv/tikv into checkpoint-v2

        commit ac005ba423289bf656dc27aff7087a34a0f43400
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Mon May 23 18:17:05 2022 +0800

            WIP

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit 4d0efff84f61818b1403b7d83a4be8e285889b79
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Fri May 20 19:00:12 2022 +0800

            WIP: mark electing regions

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit 10b4acb
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Fri May 20 13:02:26 2022 +0800

            using checkpoint v2

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit 929ea27
        Merge: e336ec6 4703705
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Thu May 19 21:14:12 2022 +0800

            Merge branch 'master' of https://github.com/tikv/tikv into inflight-detect

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit e336ec6
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Thu May 19 21:12:50 2022 +0800

            wait longer for flushing·

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit b5abdf8
        Merge: 4c600ac 8c02d7b
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Thu May 19 17:48:24 2022 +0800

            Merge branch 'master' of https://github.com/tikv/tikv into inflight-detect

        commit 4c600ac
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Thu May 19 17:18:14 2022 +0800

            fix? test

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit d9430da
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Thu May 19 16:00:22 2022 +0800

            fix? test

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit 33c8b8d
        Merge: 41c6e5a 500c4d6
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Thu May 19 14:12:18 2022 +0800

            Merge branch 'inflight-detect' of https://github.com/yujuncen/tikv into inflight-detect

        commit 41c6e5a
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Thu May 19 14:12:03 2022 +0800

            fix status transform

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit 500c4d6
        Merge: 4cf0b17 486761a
        Author: 山岚 <36239017+YuJuncen@users.noreply.github.com>
        Date:   Thu May 19 13:30:49 2022 +0800

            Merge branch 'master' into inflight-detect

        commit 4cf0b17
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Thu May 19 01:05:37 2022 +0800

            fix bad safepoint location

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit fbaf668
        Merge: b390173 9c78e8b
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Thu May 19 01:04:43 2022 +0800

            Merge branch 'master' of https://github.com/tikv/tikv into inflight-detect

        commit b390173
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Wed May 18 22:53:39 2022 +0800

            added test

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit f05af56
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Wed May 18 18:01:08 2022 +0800

            fix inf waiting

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit 363d264
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Wed May 18 16:47:11 2022 +0800

            don't advance checkpoint ts if there are inflight StartObserve

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

    commit cc70212
    Author: Yu Juncen <yujuncen@pingcap.com>
    Date:   Tue May 31 14:27:34 2022 +0800

        Squashed commit of the following:

        commit 605ec49e3686352903fb8627213f1faafe446a96
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Sat May 28 19:56:30 2022 +0800

            fix build

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit ce046e0
        Merge: 9c6d827 4facb62
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Sat May 28 19:51:09 2022 +0800

            Merge branch 'master' of https://github.com/tikv/tikv into refactor-region-op

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit 9c6d827
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Mon May 16 18:59:53 2022 +0800

            remove file on drop

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit d3408c1
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Mon May 16 17:54:01 2022 +0800

            set checkpoint to global checkpoint once start

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit c209007
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Mon May 16 17:10:32 2022 +0800

            fix tests

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit 1569fa6
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Mon May 16 15:09:10 2022 +0800

            refactor region op

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit 09edd39
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Sun May 15 14:08:19 2022 +0800

            fix scan pool name

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit ac31fb4
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Sun May 15 14:01:24 2022 +0800

            tweaking metrics

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit e11eb3d
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Sun May 15 13:25:57 2022 +0800

            add some metrics

            - Task Status
            - Internal Task (Actor Messages) Handling time

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit a9d2bfa
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Sat May 14 23:09:59 2022 +0800

            address comments

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit 9bc5605
        Merge: 3faeeb2 744d688
        Author: 山岚 <36239017+YuJuncen@users.noreply.github.com>
        Date:   Sat May 14 16:25:50 2022 +0800

            Merge branch 'master' into fix-test

        commit 3faeeb2
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Sat May 14 15:47:41 2022 +0800

            check observer version when accepting new events.

            For log backup, it is ok to repeat data, but when lock repeated when observe changing, it may make lock failed to be unlocked.

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit b746aa1
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Sat May 14 11:07:03 2022 +0800

            make clippy happy

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit 3fec0d7
        Merge: ef44a4d de208b5
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Sat May 14 10:38:43 2022 +0800

            Merge branch 'master' of https://github.com/tikv/tikv into fix-test

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit ef44a4d
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Fri May 13 23:35:56 2022 +0800

            add delay when start

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        commit 647810f
        Author: Yu Juncen <yujuncen@pingcap.com>
        Date:   Fri May 13 14:54:44 2022 +0800

            make test more stable && handle epoch not match

            Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

        Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

    Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

commit 1eca46b
Author: Yu Juncen <yujuncen@pingcap.com>
Date:   Thu Jun 2 23:45:08 2022 +0800

    make foramt

    Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

commit 94d6e0b
Author: Yu Juncen <yujuncen@pingcap.com>
Date:   Thu Jun 2 20:18:14 2022 +0800

    implement checkpoint v2

    Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

commit 0e445f8
Merge: 835436c 1698814
Author: Zak Zhao <57036248+joccau@users.noreply.github.com>
Date:   Sat Jun 11 12:19:25 2022 +0800

    Merge pull request tikv#4 from joccau/cherry-pick-12712

    security, server: fix running local test (tikv#12712)

commit 1698814
Author: 山岚 <36239017+YuJuncen@users.noreply.github.com>
Date:   Wed Jun 1 18:36:28 2022 +0800

    security, server: fix running local test (tikv#12712)

    close tikv#12711

    Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

    Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

commit 835436c
Merge: a4580f8 621c9de
Author: Zak Zhao <57036248+joccau@users.noreply.github.com>
Date:   Sat Jun 11 12:11:26 2022 +0800

    Merge pull request tikv#3 from joccau/cherry-pick-12757

    br: log-backup: fix bug about restoring point at TiCloud with KMS tikv#12757

commit a4580f8
Merge: d769651 4a29d32
Author: Zak Zhao <57036248+joccau@users.noreply.github.com>
Date:   Sat Jun 11 12:11:08 2022 +0800

    Merge pull request tikv#2 from joccau/cherry-pick-12710

    br: log backup: disable test log by default (tikv#12710)

commit d769651
Merge: c26134b ceb547e
Author: Zak Zhao <57036248+joccau@users.noreply.github.com>
Date:   Sat Jun 11 12:10:35 2022 +0800

    Merge pull request tikv#1 from joccau/cherry-pick-12640

    br: log-backup: adapt user-defined TLS config (tikv#12640)

commit 621c9de
Author: Zak Zhao <57036248+joccau@users.noreply.github.com>
Date:   Tue Jun 7 18:16:30 2022 +0800

    log-backup: fix bug about restoring point at TiCloud with KMS (tikv#12757)

    close tikv#12750, close tikv#12751

    Signed-off-by: joccau <zak.zhao@pingcap.com>

commit 4a29d32
Author: 山岚 <36239017+YuJuncen@users.noreply.github.com>
Date:   Wed Jun 1 00:42:27 2022 +0800

    log backup: disable test log by default (tikv#12710)

    close tikv#12709

    Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

commit ceb547e
Author: 山岚 <36239017+YuJuncen@users.noreply.github.com>
Date:   Mon May 30 16:52:27 2022 +0800

    log-backup: adapt user-defined TLS config (tikv#12640)

    close tikv#12668

    log-backup: adapt user-defined TLS config

    Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

    Co-authored-by: kennytm <kennytm@gmail.com>

Signed-off-by: WangLe1321 <wangle1321@163.com>
3pointer pushed a commit to 3pointer/tikv that referenced this pull request Jul 7, 2022
close tikv#12668

log-backup: adapt user-defined TLS config

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

Co-authored-by: kennytm <kennytm@gmail.com>

log backup: disable test log by default (tikv#12710)

close tikv#12709

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

log-backup: fix bug about restoring point at TiCloud with KMS (tikv#12757)

close tikv#12750, close tikv#12751

Signed-off-by: joccau <zak.zhao@pingcap.com>

security, server: fix running local test (tikv#12712)

close tikv#12711

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

implement checkpoint v2

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

make foramt

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

fix build && make clippy happy

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

fix build && make clippy happy

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

update flush time if succeeding in do_flush, ref issue#12802

Signed-off-by: joccau <zak.zhao@pingcap.com>

implement basic checkpoint v3

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

checkpoint v3 option

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

write global checkpoint ts to file if v3 is enabled

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

fix region count && don't block when responsing

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

added more log

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

added more tests

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

address comments

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

br: remove metrics when task stopped

feat: check result after remove metric

br: modify to recieve returned value

*: update max ts

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

limit the fluent when write kv file, ref issues#12869

Signed-off-by: joccau <zak.zhao@pingcap.com>

add unit test

Signed-off-by: joccau <zak.zhao@pingcap.com>

move local variables to const

Signed-off-by: joccau <zak.zhao@pingcap.com>

maintain test case, ref issues#12869

Signed-off-by: joccau <zak.zhao@pingcap.com>

Store log files by date

Signed-off-by: joccau <zak.zhao@pingcap.com>

keep advanced checkpoint-ts as safe point with ttl 12 hours, ref tikv#12903

Signed-off-by: joccau <zak.zhao@pingcap.com>

add comments

Signed-off-by: joccau <zak.zhao@pingcap.com>

added the stopped flag

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

add tests

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

make clippy happy

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

added flow control for initial scanning

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

make clippy happy

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

set ratelimit to inf when 0

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

Update endpoint.rs

fix build

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

make clippy happy

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

don't restore RollBack

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

added cfg(test) for testing

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

do not move new adding files to flushing_files if the do_flush is retry operation

Signed-off-by: joccau <zak.zhao@pingcap.com>

Store log files by date

Signed-off-by: joccau <zak.zhao@pingcap.com>

set min_begin_ts into kv file info

Signed-off-by: joccau <zak.zhao@pingcap.com>

skip empty value when parse value

Signed-off-by: joccau <zak.zhao@pingcap.com>

set service-safe-point with global-checkpoint-ts - 1

Signed-off-by: joccau <zak.zhao@pingcap.com>

add test case

Signed-off-by: joccau <zak.zhao@pingcap.com>

make clippy happy

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

Update components/backup-stream/src/checkpoint_manager.rs

Signed-off-by: 3pointer <luancheng@pingcap.com>
3pointer pushed a commit to 3pointer/tikv that referenced this pull request Jul 7, 2022
close tikv#12668

log-backup: adapt user-defined TLS config

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

Co-authored-by: kennytm <kennytm@gmail.com>

log backup: disable test log by default (tikv#12710)

close tikv#12709

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

log-backup: fix bug about restoring point at TiCloud with KMS (tikv#12757)

close tikv#12750, close tikv#12751

Signed-off-by: joccau <zak.zhao@pingcap.com>

security, server: fix running local test (tikv#12712)

close tikv#12711

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

implement checkpoint v2

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

make foramt

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

fix build && make clippy happy

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

fix build && make clippy happy

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

update flush time if succeeding in do_flush, ref issue#12802

Signed-off-by: joccau <zak.zhao@pingcap.com>

implement basic checkpoint v3

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

checkpoint v3 option

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

write global checkpoint ts to file if v3 is enabled

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

fix region count && don't block when responsing

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

added more log

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

added more tests

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

address comments

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

br: remove metrics when task stopped

feat: check result after remove metric

br: modify to recieve returned value

*: update max ts

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

limit the fluent when write kv file, ref issues#12869

Signed-off-by: joccau <zak.zhao@pingcap.com>

add unit test

Signed-off-by: joccau <zak.zhao@pingcap.com>

move local variables to const

Signed-off-by: joccau <zak.zhao@pingcap.com>

maintain test case, ref issues#12869

Signed-off-by: joccau <zak.zhao@pingcap.com>

Store log files by date

Signed-off-by: joccau <zak.zhao@pingcap.com>

keep advanced checkpoint-ts as safe point with ttl 12 hours, ref tikv#12903

Signed-off-by: joccau <zak.zhao@pingcap.com>

add comments

Signed-off-by: joccau <zak.zhao@pingcap.com>

added the stopped flag

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

add tests

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

make clippy happy

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

added flow control for initial scanning

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

make clippy happy

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

set ratelimit to inf when 0

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

Update endpoint.rs

fix build

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

make clippy happy

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

don't restore RollBack

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

added cfg(test) for testing

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

do not move new adding files to flushing_files if the do_flush is retry operation

Signed-off-by: joccau <zak.zhao@pingcap.com>

Store log files by date

Signed-off-by: joccau <zak.zhao@pingcap.com>

set min_begin_ts into kv file info

Signed-off-by: joccau <zak.zhao@pingcap.com>

skip empty value when parse value

Signed-off-by: joccau <zak.zhao@pingcap.com>

set service-safe-point with global-checkpoint-ts - 1

Signed-off-by: joccau <zak.zhao@pingcap.com>

add test case

Signed-off-by: joccau <zak.zhao@pingcap.com>

make clippy happy

Signed-off-by: Yu Juncen <yujuncen@pingcap.com>

Update components/backup-stream/src/checkpoint_manager.rs

Signed-off-by: 3pointer <luancheng@pingcap.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution Type: PR - From contributors needs-cherry-pick-release-6.1 release-note size/XXL status/can-merge Status: Can merge to base branch status/LGT2 Status: PR - There are already 2 approvals
Projects
None yet
Development

Successfully merging this pull request may close these issues.

log-bakcup: support custom TLS
5 participants