Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

badgerengine doesn't release disk space since badger gc doesn't work. #454

Closed
crazycs520 opened this issue Mar 28, 2022 · 3 comments
Closed
Labels
bug Something isn't working

Comments

@crazycs520
Copy link

crazycs520 commented Mar 28, 2022

What version of Genji are you using?

$ genji version
v0.14.1

What did you do?

Related issue: https://discuss.dgraph.io/t/badgerdb-consume-too-much-disk-space/17070/2

This problem is caused when Transaction Commit, the Transaction doesn't call tx.Discard() then cause badger gc doesn't work. Since the badger API said:

// When you create a new transaction, it is absolutely essential to call
// Discard(). This should be done irrespective of what the update param is set
// to. Commit API internally runs Discard, but running it twice wouldn't cause
// any issues.
//
//  txn := db.NewTransaction(false)
//  defer txn.Discard()
//  // Call various APIs.
func (db *DB) NewTransaction(update bool) *Txn {
	return db.newTransaction(update, false)
}

But badgerengine doesn't call txn.Discard() before txn.Commit

image

What did you expect to see?

Release disk space after I delete data and badger gc finish.

What did you see instead?

genjidb with badgerdb consume too much disk space and badger gc doesn't work.

@crazycs520 crazycs520 added the bug Something isn't working label Mar 28, 2022
@crazycs520
Copy link
Author

crazycs520 commented Mar 28, 2022

41729dc

I have bug-fix PR, but look like badgerengine has been removed.

@asdine
Copy link
Collaborator

asdine commented Mar 28, 2022

This is solved by calling Rollback after Commit, the same way as Badger:

defer tx.Rollback()
...
return tx.Commit()

The issue is that Rollback documentation is not clear enough, it must definitely be improved.

but look like badgerengine has been removed.

Yes, It has been removed for several reasons:

  • Badger doesn't seem to be actively developed for the moment: https://github.com/dgraph-io/badger/graphs/commit-activity
  • Dgraph has been having some financial issues these past months so it's not clear if Badger is a priority
  • Genji is gonna come with it's own MVCC implementation, and Pebble seems to be the perfect choice for the backend

Genji is still a work in progress, so these kind of architectural changes are expected to happen until it reaches v1 unfortunately.

@asdine
Copy link
Collaborator

asdine commented Jun 19, 2022

Closing this as Badger is no longer supported and the new release will be published soon

@asdine asdine closed this as completed Jun 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants