Skip to content
This repository has been archived by the owner on Apr 1, 2024. It is now read-only.

ISSUE-4632: Bookie GC doesn't clean up overreplicated ledgers from entry logs #183

Closed
sijie opened this issue Dec 27, 2019 · 0 comments
Closed
Labels

Comments

@sijie
Copy link
Member

sijie commented Dec 27, 2019

Original Issue: apache#4632


Motivation

  • Sometimes due to overreplication, bookie contains ledgers which are not owned by that bookie anymore and that bookie is not part of the ensemble-list of those ledgers. In this case, GC finds out those overreplicated ledgers and
  • deletes their index from dbStorage (rocksDB) and
  • tries to delete them from entrylog files.

However, bookie doesn't delete them from entry-log files due to change made in #870 where bookie avoids deleting ledger if znode of that ledger exists.

Because of that bookie ends up storing large number entrylog files with ledgers which are owned by different bookies. It also cause OOM when GC tries to deal with large number of entry log files.

Fix

  1. OOM should be addressed by: 1949 or 1938
  2. And clean up overreplicated ledgers which are owned by different bookies should be fixed by this commit
    I will create a PR with this fix.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

1 participant