Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: Size based retention #7927

Open
wants to merge 74 commits into
base: main
Choose a base branch
from
Open

WIP: Size based retention #7927

wants to merge 74 commits into from
+594 −71

Commits on Dec 12, 2022

  1. Droping the first lines of code... Just seeing how it could be

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    30fe91a View commit details
    Browse the repository at this point in the history

  2. Constant added and if... changed

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    664977f View commit details
    Browse the repository at this point in the history

  3. DiskStatus struct and DiskUsage function added

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    c1a6b10 View commit details
    Browse the repository at this point in the history

  4. First lines of DeleteChunksBasedOnBlockSize

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    3542372 View commit details
    Browse the repository at this point in the history

  5. Missing "(" and ")"

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    2647d1f View commit details
    Browse the repository at this point in the history

  6. First unit test written in golang

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    6842ff5 View commit details
    Browse the repository at this point in the history

  7. Implement the deletion process

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    c5c74f2 View commit details
    Browse the repository at this point in the history

  8. Refactor bytesToDelete function

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    ae6dd9f View commit details
    Browse the repository at this point in the history

  9. Move DiskStatus struct to utils

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    0cc8d44 View commit details
    Browse the repository at this point in the history

  10. Move DiskStatus to a better place (?)

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    b44fe34 View commit details
    Browse the repository at this point in the history

  11. Add FIXME

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    85a72ec View commit details
    Browse the repository at this point in the history

  12. bucketBlockSizeRetentionPercentaje -> bucketBlockSizeRetentionInterval

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    abf6f37 View commit details
    Browse the repository at this point in the history

  13. Improve FIXME comment

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    4b8e50d View commit details
    Browse the repository at this point in the history

  14. Some refactor and encasuplation

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    7b85a62 View commit details
    Browse the repository at this point in the history

  15. Adding SizeBasedRetentionPercentage to common.go

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    1dad70c View commit details
    Browse the repository at this point in the history

  16. There are still a lot of thinks to fix, but it is "working"

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    3001053 View commit details
    Browse the repository at this point in the history

  17. Fix TODO comment

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    b73bdf0 View commit details
    Browse the repository at this point in the history

  18. Check if we are using boltdb as storage index

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    ee75bc0 View commit details
    Browse the repository at this point in the history

  19. Fix comment to help us solve the issue

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    8dc931d View commit details
    Browse the repository at this point in the history

  20. NewCompactor modified. Now receives fsConfig

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    b0d5a42 View commit details
    Browse the repository at this point in the history

  21. Refactor DeleteChunksBasedOnBlockSize, remove from FSObjectClient, ad… 

    …d test
    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    46688f9 View commit details
    Browse the repository at this point in the history

  22. missing code in merge

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    be4cfb3 View commit details
    Browse the repository at this point in the history

  23. fixing import in test

    @Abuelodelanada
    Abuelodelanada committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    2f75e1b View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2022

  1. Merge branch 'main' into size_based_retention

    @Abuelodelanada
    Abuelodelanada committed Dec 14, 2022
    Configuration menu
    Copy the full SHA
    810d2b0 View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2023

  1. remove duplicated CompactorRing

    @Abuelodelanada
    Abuelodelanada committed Jan 10, 2023
    Configuration menu
    Copy the full SHA
    538ba3a View commit details
    Browse the repository at this point in the history

  2. removed not used imports

    @Abuelodelanada
    Abuelodelanada committed Jan 10, 2023
    Configuration menu
    Copy the full SHA
    003cf1f View commit details
    Browse the repository at this point in the history

  3. add missing arg to NewCompactor

    @Abuelodelanada
    Abuelodelanada committed Jan 10, 2023
    Configuration menu
    Copy the full SHA
    89fc6ce View commit details
    Browse the repository at this point in the history

  4. fixing tests

    @Abuelodelanada
    Abuelodelanada committed Jan 10, 2023
    Configuration menu
    Copy the full SHA
    261ee83 View commit details
    Browse the repository at this point in the history

  5. linting common.go

    @Abuelodelanada
    Abuelodelanada committed Jan 10, 2023
    Configuration menu
    Copy the full SHA
    a3aaea3 View commit details
    Browse the repository at this point in the history

  6. update documentation

    @Abuelodelanada
    Abuelodelanada committed Jan 10, 2023
    Configuration menu
    Copy the full SHA
    22d7c8b View commit details
    Browse the repository at this point in the history

  7. replace deprecated io/ioutil with os

    @Abuelodelanada
    Abuelodelanada committed Jan 10, 2023
    Configuration menu
    Copy the full SHA
    ecb1776 View commit details
    Browse the repository at this point in the history

  8. more linting fixes

    @Abuelodelanada
    Abuelodelanada committed Jan 10, 2023
    Configuration menu
    Copy the full SHA
    e368e75 View commit details
    Browse the repository at this point in the history

  9. replace another deprecated io/ioutil with os

    @Abuelodelanada
    Abuelodelanada committed Jan 10, 2023
    Configuration menu
    Copy the full SHA
    42370b6 View commit details
    Browse the repository at this point in the history

Commits on Jan 11, 2023

  1. os.ReadDir behaves differently than ioutils.ReadDir

    @Abuelodelanada
    Abuelodelanada committed Jan 11, 2023
    Configuration menu
    Copy the full SHA
    ea68bd8 View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2023

  1. Fix lang convention error -> err

    @Abuelodelanada
    Abuelodelanada committed Jan 18, 2023
    Configuration menu
    Copy the full SHA
    641c03e View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2023

  1. Merge branch 'main' into size_based_retention

    @Abuelodelanada
    Abuelodelanada committed Jan 23, 2023
    Configuration menu
    Copy the full SHA
    d26d03e View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2023

  1. WIP

    @MasslessParticle
    MasslessParticle committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    04f32df View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'canonical/size_based_retention' into tp… 

    …atterson/space-compaction
    @MasslessParticle
    MasslessParticle committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    ce96f19 View commit details
    Browse the repository at this point in the history

  3. WIP

    @MasslessParticle
    MasslessParticle committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    99a06e3 View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2023

  1. Merge branch 'main' into size_based_retention

    @rbarry82
    rbarry82 committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    5879265 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/tpatterson/space-compaction' i… 

    …nto size_based_retention
    @rbarry82
    rbarry82 committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    38f5e66 View commit details
    Browse the repository at this point in the history

  3. Finish up renaming after merge to continue

    @rbarry82
    rbarry82 committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    1c6cf85 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into size_based_retention

    @rbarry82
    rbarry82 committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    1827d50 View commit details
    Browse the repository at this point in the history

  5. Re-merge

    @rbarry82
    rbarry82 committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    b39ad06 View commit details
    Browse the repository at this point in the history

  6. fix ineffectual assignment to err

    @Abuelodelanada
    Abuelodelanada committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    7bff238 View commit details
    Browse the repository at this point in the history

Commits on Mar 31, 2023

  1. refactor functions

    @Abuelodelanada
    Abuelodelanada committed Mar 31, 2023
    Configuration menu
    Copy the full SHA
    1ed4fea View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2023

  1. Go through indexes to get chunkrefs

    @rbarry82
    rbarry82 committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    32f6492 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2023

  1. Write marker entries for chunks

    @rbarry82
    rbarry82 committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    9916f09 View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2023

  1. Finish retention

    @rbarry82
    rbarry82 committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    734ae82 View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2023

  1. Merge pull request #3 from canonical/use-compacted-index 

    Go through indexes to get chunkrefs
    @rbarry82
    rbarry82 committed Apr 10, 2023
    Configuration menu
    Copy the full SHA
    e40ecb0 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2023

  1. Extract/refactor

    @rbarry82
    rbarry82 committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    c0e7746 View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2023

  1. Catch empty indexes and let them be cleaned up

    @rbarry82
    rbarry82 committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    c06984c View commit details
    Browse the repository at this point in the history

  2. Don't mark twice

    @rbarry82
    rbarry82 committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    eeb151f View commit details
    Browse the repository at this point in the history

  3. adding and improving log

    @Abuelodelanada
    Abuelodelanada committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    a844dde View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2023

  1. Merge branch 'main' into size_based_retention

    @Abuelodelanada
    Abuelodelanada committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    7c716a1 View commit details
    Browse the repository at this point in the history

  2. Bubble up a reasonable error from ChunkIterator

    @rbarry82
    rbarry82 committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    716fcd1 View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2023

  1. change log level

    @Abuelodelanada
    Abuelodelanada committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    794b501 View commit details
    Browse the repository at this point in the history

  2. remove functions that were re-refactored

    @Abuelodelanada
    Abuelodelanada committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    d538b7d View commit details
    Browse the repository at this point in the history

Commits on May 5, 2023

  1. Merge branch 'main' into size_based_retention

    @Abuelodelanada
    Abuelodelanada committed May 5, 2023
    Configuration menu
    Copy the full SHA
    6c2e2e9 View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2023

  1. Merge branch 'main' into size_based_retention

    @Abuelodelanada
    Abuelodelanada committed Jun 13, 2023
    Configuration menu
    Copy the full SHA
    2d5c863 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2023

  1. Merge branch 'main' into size_based_retention

    @Abuelodelanada
    Abuelodelanada committed Jun 26, 2023
    Configuration menu
    Copy the full SHA
    2073945 View commit details
    Browse the repository at this point in the history

  2. remove commented code

    @Abuelodelanada
    Abuelodelanada committed Jun 26, 2023
    Configuration menu
    Copy the full SHA
    ba5c932 View commit details
    Browse the repository at this point in the history

  3. remove turning off index compaction

    @Abuelodelanada
    Abuelodelanada committed Jun 26, 2023
    Configuration menu
    Copy the full SHA
    dd77bd7 View commit details
    Browse the repository at this point in the history

  4. remove nesting, add guards

    @Abuelodelanada
    Abuelodelanada committed Jun 26, 2023
    Configuration menu
    Copy the full SHA
    a05a5d8 View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2023

  1. Merge branch 'grafana:main' into size_based_retention

    @Abuelodelanada
    Abuelodelanada committed Jun 27, 2023
    Configuration menu
    Copy the full SHA
    69f78be View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2023

  1. fix NewCompactor func signature

    @Abuelodelanada
    Abuelodelanada committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    1fbc41b View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2023

  1. remove not used function

    @Abuelodelanada
    Abuelodelanada committed Jul 4, 2023
    Configuration menu
    Copy the full SHA
    4ac5034 View commit details
    Browse the repository at this point in the history

  2. fix NewCompactor() call

    @Abuelodelanada
    Abuelodelanada committed Jul 4, 2023
    Configuration menu
    Copy the full SHA
    53156e0 View commit details
    Browse the repository at this point in the history

  3. make MarkerMetrics public

    @Abuelodelanada
    Abuelodelanada committed Jul 4, 2023
    Configuration menu
    Copy the full SHA
    af0b78c View commit details
    Browse the repository at this point in the history

  4. Refactor NewMarker. SizeBasedRetentionCleanerService added

    @Abuelodelanada
    Abuelodelanada committed Jul 4, 2023
    Configuration menu
    Copy the full SHA
    bedd990 View commit details
    Browse the repository at this point in the history

  5. compactor refactored

    @Abuelodelanada
    Abuelodelanada committed Jul 4, 2023
    Configuration menu
    Copy the full SHA
    dcf45cc View commit details
    Browse the repository at this point in the history

  6. Fix Test_Retention

    @Abuelodelanada
    Abuelodelanada committed Jul 4, 2023
    Configuration menu
    Copy the full SHA
    797fea8 View commit details
    Browse the repository at this point in the history

  7. make NewMarkerMetrics public

    @Abuelodelanada
    Abuelodelanada committed Jul 4, 2023
    Configuration menu
    Copy the full SHA
    83cc6ec View commit details
    Browse the repository at this point in the history

  8. filling c.indexStorageClient

    @Abuelodelanada
    Abuelodelanada committed Jul 4, 2023
    Configuration menu
    Copy the full SHA
    b5abb85 View commit details
    Browse the repository at this point in the history