Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add BadgerDB driver #16

Open
narqo opened this issue Dec 15, 2019 · 4 comments
Open

Add BadgerDB driver #16

narqo opened this issue Dec 15, 2019 · 4 comments

Comments

@narqo
Copy link

narqo commented Dec 15, 2019

BadgerDB is an embeddable, fast, pure-Go key-value database. At least from a brief look, it feels that it might fit well as an alternative to sqlite:

  • it's pure Go, meaning it doesn't (necessarily) require cgo, which looks attractive in cases similar to described in Stub-out sqlite drivers to build Kine without CGO #14
  • it supports concurrent ACID transactions (I haven't dig deep into the implementation of logstructured.Log, so can't say if that's important for a kine's driver).

See "Design" secsion of BadgerDB's README.

Note, BadgerDB v2 uses optional zstd compression, which currently requires cgo. Refer to dgraph-io/badger#1162 on switching to pure Go zstd.

Nevertheless, I don't think zstd level of compression is required for kine/k3s cases.

@ibuildthecloud
Copy link
Contributor

This sounds interesting. Some things to consider first. Kine exposes an etcd style abstracted on (right now) SQL backends. Even though etcd is a key value store, it's actually hard to reproduce the behavior of etcd on an arbitrary k/v (at least I haven't figured it out yet). The reason is that Kubernetes is very tied to the way etcd works with revision counters. There is a basic assumption that a historical record of changes is available and that one can query them for a short period of time. So internally what you want out of a data store is to be able to store a log of changes and then to be able to query that log easily. So in SQL it's quite easy to do. I haven't thought of a simple approach in a k/v store yet but also haven't given a lot of thought to it. Obviously it's possible, just might be a bit more work than one would expect.

Also, this code base is very poorly documented, sorry.

@narqo
Copy link
Author

narqo commented Dec 19, 2019

I've spent some time reading about etcd's data-model; indeed it's a bit more complex than I expected after looking at sqlite's driver but still doable. Will try to sketch up a PR.

@carlosedp
Copy link
Contributor

What about using bbolt directly (https://github.com/etcd-io/bbolt) as an embedded DB? Dunno if it would work... just a thought :)

@emolitor
Copy link

emolitor commented Aug 4, 2021

Minor note, the latest version of BadgerDB, from v3.2103.1 onward, is now using a pure go zstd dependency.

solmyrcan pushed a commit to solmyrcan/kine that referenced this issue Mar 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants