Avoid index (epoch) cleanup and compaction during index reads #3638
Comments
julio-lopez
added
performance
enterprise
|
Proposed approach
|
|
Index loading simplified call graph before the proposed change
|
Refactor: move index compaction and cleanup out of refreshAttemptLocked Introduces an `allowWritesOnLoadHelper` to check whether or not writes can be performed when loading the indexes. Currently this is only a function of whether the storage is in read-only mode. In the near future, an explicit flag will be added to control this behavior. Fix epoch manager: avoid single-epoch compaction when writes are disallowed. Functional change: prevents compacting single epochs when writes are disallowed, that is when using read-only storage. Currently, the epoch manager will attempt to perform single-epoch compactions for all eligible epochs, even on read-only storage. Ref: - #3224 - #3225 - #3638 - #3639
|
silly question, but I could use a little extra context: what type(s) of maintenance will allow index compaction? Will it be full and quick maintenance or just one of them? |
Conditionally disables epoch index maintenance operations when loading indexes. This prevents (potentially expensive) cleanup write operations on the index read path. The behavior is controlled via the `epoch.Manager.allowCleanupWritesOnIndexLoad` field, which can be temporarily overridden via an environment variable. This override mechanism will be removed in the near future. Refs: - #3174 - #3224 - #3225 - #3638 - #3639
@ashmrtn Good question. The approach implemented in the current PR is as follows. For quick maintenance:
For full maintenance:
Does that make sense? |
|
Thanks for the info Julio! The details you laid out make sense from an understanding perspective However, looking at the additional info I'm unsure how to feel about the range compaction being in full maintenance. I don't have hard numbers, but I feel like it starts to turn the index compaction/data blob compaction into a bit of a balancing act. Keeping the index compacted with range compactions helps reduce GETs and general load time to do any work on the repo. However, if it's with full maintenance, which also does data blob compaction, power users need to consider the cost of potentially compacting data blobs which could trigger GET requests to slower storage tiers Do you have any thoughts/guidance on this? |
|
@ashmrtn The change should not adversely impact users, and instead in some cases result in resource savings, potentially including fewer storage requests (GETs, etc). Overall, this does not change the effective frequency of single epoch or range compactions. See details below. Here's the reasoning, based also on observations with various repositories:
Does that make sense? Does that answer your question? |
Extracted from kopia#3651. Thanks to @plar and @redgoat650 for the suggestions. Ref: - kopia#3603 - kopia#3645 - kopia#3638 - kopia#3639
Refactor: move index compaction and cleanup out of refreshAttemptLocked Introduces an `allowWritesOnLoadHelper` to check whether or not writes can be performed when loading the indexes. Currently this is only a function of whether the storage is in read-only mode. In the near future, an explicit flag will be added to control this behavior. Fix epoch manager: avoid single-epoch compaction when writes are disallowed. Functional change: prevents compacting single epochs when writes are disallowed, that is when using read-only storage. Currently, the epoch manager will attempt to perform single-epoch compactions for all eligible epochs, even on read-only storage. Ref: - kopia#3224 - kopia#3225 - kopia#3638 - kopia#3639
…#3645) Conditionally disables epoch index maintenance operations when loading indexes. This prevents (potentially expensive) cleanup write operations on the index read path. The behavior is controlled via the `epoch.Manager.allowCleanupWritesOnIndexLoad` field, which can be temporarily overridden via an environment variable. This override mechanism will be removed in the near future. Refs: - kopia#3174 - kopia#3224 - kopia#3225 - kopia#3638 - kopia#3639
Refactoring for the original implementation with intRange and getKeyRange from closed-open ranges [lo, hi) to closed ranges: [lo, hi]. The primary motivation is for consistency with the implementation of epoch.RangeMetadata in the same package, and thus avoid confusion and reduce cognitive load. Changes: - adds a getContiguousKeyRange wrapper that checks for contiguity. - getKeyRange simply returns a range with minimum and maximum values for the keys in the map. - changes the range implementation from closed-open ranges [lo, hi) to closed ranges: [lo, hi] where both lo and hi are included in the range. - Additional unit tests are included. - renames intRange to closedIntRange to reflect new functionality. Ref: - Follow up refactor(general): add epoch.getKeyRange helper #3721 - Needed for refactor(general): add epoch.Manager.MaybeCompactSingleEpoch #3728 - Avoid index (epoch) cleanup and compaction during index reads #3638
Add: - epoch.Manager.MaybeCompactSingleEpoch - getCompactedEpochRange helper - oldestUncompactedEpoch helper - TestOldestUncompactedEpoch - Tests for MaybeCompactSingleEpoch Ref: - Subset and dependency of #3651 - Depends on #3735 - Avoid index (epoch) cleanup and compaction during index reads #3638 - Make "read" commands/operations really read-only. #3639
Refactor: move index compaction and cleanup out of refreshAttemptLocked Introduces an `allowWritesOnLoadHelper` to check whether or not writes can be performed when loading the indexes. Currently this is only a function of whether the storage is in read-only mode. In the near future, an explicit flag will be added to control this behavior. Fix epoch manager: avoid single-epoch compaction when writes are disallowed. Functional change: prevents compacting single epochs when writes are disallowed, that is when using read-only storage. Currently, the epoch manager will attempt to perform single-epoch compactions for all eligible epochs, even on read-only storage. Ref: - kopia#3224 - kopia#3225 - kopia#3638 - kopia#3639
…#3645) Conditionally disables epoch index maintenance operations when loading indexes. This prevents (potentially expensive) cleanup write operations on the index read path. The behavior is controlled via the `epoch.Manager.allowCleanupWritesOnIndexLoad` field, which can be temporarily overridden via an environment variable. This override mechanism will be removed in the near future. Refs: - kopia#3174 - kopia#3224 - kopia#3225 - kopia#3638 - kopia#3639
…3651) Perform index epoch compaction and cleanup during repository maintenance * refactor: rename maintenance task for deleting superseded indexes * maintenance task to cleanup epoch markers * maintenance task to advance write index epoch * maintenance task to compact epoch ranges * quick maintenance task to compact a single (index) epoch * full maintenance task to compact a single (index) epoch Ref: - #3638 - #3639 Followup to: - #3603 - #3645 Related helpers and changes: - #3721 - #3735 - #3709 - #3728 - #3727 - #3726
…opia#3651) Perform index epoch compaction and cleanup during repository maintenance * refactor: rename maintenance task for deleting superseded indexes * maintenance task to cleanup epoch markers * maintenance task to advance write index epoch * maintenance task to compact epoch ranges * quick maintenance task to compact a single (index) epoch * full maintenance task to compact a single (index) epoch Ref: - kopia#3638 - kopia#3639 Followup to: - kopia#3603 - kopia#3645 Related helpers and changes: - kopia#3721 - kopia#3735 - kopia#3709 - kopia#3728 - kopia#3727 - kopia#3726
…ch_CompactionError (kopia#3755) Ref: - kopia#3638
Objectives
Current Issues:
Currently, kopia is performing writes on the index load path.
repository connectorsnapshot listamong others.The text was updated successfully, but these errors were encountered: