Show file tree
Hide file tree
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersThis file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Please sign in to comment.
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[BK-SERVER] Clean up over-replicated ledgers owned by different bookies
### Motivation As described at: apache/pulsar#4632 - Sometimes due to overreplication, bookie contains ledgers which are not owned by that bookie anymore and that bookie is not part of the ensemble-list of those ledgers. In this case, GC finds out those overreplicated ledgers and - deletes their index from dbStorage (rocksDB) and - tries to delete them from entrylog files. However, bookie doesn't delete them from entry-log files due to change made in [#870](#870) where bookie avoids deleting ledger if znode of that ledger exists. Because of that bookie ends up storing large number entrylog files with ledgers which are owned by different bookies. It also cause OOM when GC tries to deal with large number of entry log files. ### Modification Delete the ledgers if bookie is not part of ensemble list of over-replicated ledgers Reviewers: Enrico Olivelli <email@example.com>, Sijie Guo <firstname.lastname@example.org> This closes #2119 from rdhabalia/overRepl
- Loading branch information
Showing 2 changed files with 82 additions and 4 deletions.