Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

storage: Use etcd lease to manage safe point expiration #4984

Closed
wants to merge 43 commits into from
Closed

storage: Use etcd lease to manage safe point expiration #4984

wants to merge 43 commits into from

Commits on May 12, 2022

  1. added storage methods for RawKV GC 

    Signed-off-by: AmoebaProtozoa <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 12, 2022
    Configuration menu
    Copy the full SHA
    32c44ee View commit details
    Browse the repository at this point in the history

  2. push back on updating kvproto in go.mod 

    Signed-off-by: AmoebaProtozoa <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 12, 2022
    Configuration menu
    Copy the full SHA
    b12eb9b View commit details
    Browse the repository at this point in the history

  3. linting 

    Signed-off-by: AmoebaProtozoa <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 12, 2022
    Configuration menu
    Copy the full SHA
    d0b4f3b View commit details
    Browse the repository at this point in the history

  4. changed storage path structure 

    Signed-off-by: AmoebaProtozoa <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 12, 2022
    Configuration menu
    Copy the full SHA
    a2ba0e7 View commit details
    Browse the repository at this point in the history

  5. update comments 

    Signed-off-by: AmoebaProtozoa <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 12, 2022
    Configuration menu
    Copy the full SHA
    b9bb3e4 View commit details
    Browse the repository at this point in the history

  6. added ByKeySpace suffix for disambiguity 

    Signed-off-by: AmoebaProtozoa <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 12, 2022
    Configuration menu
    Copy the full SHA
    a7e3ece View commit details
    Browse the repository at this point in the history

Commits on May 13, 2022

  1. removed default key spaces 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 13, 2022
    Configuration menu
    Copy the full SHA
    7ff125e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into RawKV_GC_API_storage

    @AmoebaProtozoa
    AmoebaProtozoa committed May 13, 2022
    Configuration menu
    Copy the full SHA
    1bdb642 View commit details
    Browse the repository at this point in the history

Commits on May 16, 2022

  1. renaming, move delete expired safepoints to a goroutine 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 16, 2022
    Configuration menu
    Copy the full SHA
    1ebb51d View commit details
    Browse the repository at this point in the history

  2. update tests 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 16, 2022
    Configuration menu
    Copy the full SHA
    312704a View commit details
    Browse the repository at this point in the history

  3. lint 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 16, 2022
    Configuration menu
    Copy the full SHA
    766b344 View commit details
    Browse the repository at this point in the history

  4. added back KeySpaceGCSafePoint 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 16, 2022
    Configuration menu
    Copy the full SHA
    de77dfc View commit details
    Browse the repository at this point in the history

  5. remove expired all at once 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 16, 2022
    Configuration menu
    Copy the full SHA
    d6eda93 View commit details
    Browse the repository at this point in the history

  6. address comments 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 16, 2022
    Configuration menu
    Copy the full SHA
    cbe1ed0 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'master' into RawKV_GC_API_storage

    @AmoebaProtozoa
    AmoebaProtozoa committed May 16, 2022
    Configuration menu
    Copy the full SHA
    9b27bcd View commit details
    Browse the repository at this point in the history

  8. move sleep to failpoint 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 16, 2022
    Configuration menu
    Copy the full SHA
    88610a0 View commit details
    Browse the repository at this point in the history

Commits on May 17, 2022

  1. modified failpoint to eliminate sleep 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 17, 2022
    Configuration menu
    Copy the full SHA
    995033e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into RawKV_GC_API_storage

    @AmoebaProtozoa
    AmoebaProtozoa committed May 17, 2022
    Configuration menu
    Copy the full SHA
    3099939 View commit details
    Browse the repository at this point in the history

Commits on May 18, 2022

  1. log error when failed to remove expired service safe point 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 18, 2022
    Configuration menu
    Copy the full SHA
    a825d37 View commit details
    Browse the repository at this point in the history

  2. added load revision 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 18, 2022
    Configuration menu
    Copy the full SHA
    8bef6ee View commit details
    Browse the repository at this point in the history

  3. added SaveWithTTL to kvs 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 18, 2022
    Configuration menu
    Copy the full SHA
    89cf4d0 View commit details
    Browse the repository at this point in the history

  4. updated storage methods to use lease 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 18, 2022
    Configuration menu
    Copy the full SHA
    b8bc86f View commit details
    Browse the repository at this point in the history

  5. update tests to use ttl 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 18, 2022
    Configuration menu
    Copy the full SHA
    1eca8d9 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'master' into RawKV_GC_API_storageTTL

    @AmoebaProtozoa
    AmoebaProtozoa committed May 18, 2022
    Configuration menu
    Copy the full SHA
    7e3651e View commit details
    Browse the repository at this point in the history

Commits on May 27, 2022

  1. merge and resolve conflict 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 27, 2022
    Configuration menu
    Copy the full SHA
    78f3e14 View commit details
    Browse the repository at this point in the history

  2. lint 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 27, 2022
    Configuration menu
    Copy the full SHA
    176061e View commit details
    Browse the repository at this point in the history

  3. use etcdutil to save with ttl 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 27, 2022
    Configuration menu
    Copy the full SHA
    5d994ea View commit details
    Browse the repository at this point in the history

  4. update go mod 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 27, 2022
    Configuration menu
    Copy the full SHA
    a36305a View commit details
    Browse the repository at this point in the history

Commits on May 30, 2022

  1. Merge branch 'master' into RawKV_GC_API_storageTTL

    @AmoebaProtozoa
    AmoebaProtozoa committed May 30, 2022
    Configuration menu
    Copy the full SHA
    28a4e9e View commit details
    Browse the repository at this point in the history

Commits on May 31, 2022

  1. update go mod 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 31, 2022
    Configuration menu
    Copy the full SHA
    d8e7ab6 View commit details
    Browse the repository at this point in the history

  2. update test/client/go.mod 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed May 31, 2022
    Configuration menu
    Copy the full SHA
    6d2f26d View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2022

  1. Merge branch 'master' into RawKV_GC_API_storageTTL

    @AmoebaProtozoa
    AmoebaProtozoa committed Jun 7, 2022
    Configuration menu
    Copy the full SHA
    594edc7 View commit details
    Browse the repository at this point in the history

  2. small cleanup 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed Jun 7, 2022
    Configuration menu
    Copy the full SHA
    3ab7d4f View commit details
    Browse the repository at this point in the history

Commits on Jun 8, 2022

  1. avoid modification to general kv interface 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed Jun 8, 2022
    Configuration menu
    Copy the full SHA
    a379eea View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into RawKV_GC_API_storageTTL

    @AmoebaProtozoa
    AmoebaProtozoa committed Jun 8, 2022
    Configuration menu
    Copy the full SHA
    152fe6c View commit details
    Browse the repository at this point in the history

  3. storage: handle case where ttl is maxInt 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed Jun 8, 2022
    Configuration menu
    Copy the full SHA
    a8790d5 View commit details
    Browse the repository at this point in the history

  4. add etcdKVBase comments 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed Jun 8, 2022
    Configuration menu
    Copy the full SHA
    5269aff View commit details
    Browse the repository at this point in the history

  5. storage: Revision should be int64 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed Jun 8, 2022
    Configuration menu
    Copy the full SHA
    7c05203 View commit details
    Browse the repository at this point in the history

  6. storage: add revision tests 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed Jun 8, 2022
    Configuration menu
    Copy the full SHA
    77d6f26 View commit details
    Browse the repository at this point in the history

  7. storage: lint 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed Jun 8, 2022
    Configuration menu
    Copy the full SHA
    bda31f4 View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2022

  1. storage: change spaceID to uint32 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed Jun 14, 2022
    Configuration menu
    Copy the full SHA
    9ff8090 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2022

  1. storage: migrate tests to testify 

    Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
    @AmoebaProtozoa
    AmoebaProtozoa committed Jun 21, 2022
    Configuration menu
    Copy the full SHA
    ffe8a18 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of github.com:tikv/pd into RawKV_GC_API_storageTTL

    @AmoebaProtozoa
    AmoebaProtozoa committed Jun 21, 2022
    Configuration menu
    Copy the full SHA
    ab98f7c View commit details
    Browse the repository at this point in the history