Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

store/proxy: Deduplicate chunks on StoreAPI level. Recommend chunk sorting for StoreAPI. #2603

Merged
merged 3 commits into from
May 18, 2020
Merged

store/proxy: Deduplicate chunks on StoreAPI level. Recommend chunk sorting for StoreAPI. #2603

merged 3 commits into from
May 18, 2020

Commits on May 14, 2020

  1. Deduplicate chunk dups on proxy StoreAPI level. Recommend chunk sorti… 

    …ng for StoreAPI.

Also: Merge same series together on proxy level instead select. This allows better dedup efficiency.

Partially fixes: #2303

Cases like overlapped data from store and sidecar and 1:1 duplicates are optimized as soon as it's possible.
This case was highly visible on GitLab repro data and exists in most of Thanos setup.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
    @bwplotka
    bwplotka committed May 14, 2020
    Configuration menu
    Copy the full SHA
    5bfcbe9 View commit details
    Browse the repository at this point in the history

Commits on May 15, 2020

  1. Optimized algorithm to combine series only on start. 

    Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
    @bwplotka
    bwplotka committed May 15, 2020
    Configuration menu
    Copy the full SHA
    f6004ab View commit details
    Browse the repository at this point in the history

Commits on May 16, 2020

  1. Optimized chunk comparision for overlaps. 

    Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
    @bwplotka
    bwplotka committed May 16, 2020
    Configuration menu
    Copy the full SHA
    b6579bb View commit details
    Browse the repository at this point in the history