Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider replacing embedded sqlite with *embedded* etcd in the build #845

Closed
sandys opened this issue Sep 28, 2019 · 6 comments
Closed

Consider replacing embedded sqlite with *embedded* etcd in the build #845

sandys opened this issue Sep 28, 2019 · 6 comments

Comments

@sandys
Copy link

@sandys sandys commented Sep 28, 2019

Part of the simplicity and attraction of sqlite is that it is embedded inside k3s. That makes it very simple to use and manage.

etcd3 can also be embedded in a similar way

  1. https://godoc.org/github.com/etcd-io/etcd/embed
  2. http://docs.activestate.com/activego/1.8/pkg/github.com/coreos/etcd/embed/
  3. https://github.com/sensu/sensu-go/blob/master/backend/etcd/etcd.go

Since the HA direction needs etcd anyway.. I'm proposing dropping support for sqlite as the default embedded non-HA option and switch to embedded etcd as the default. This will reduce overall effort of maintainability of two entirely different datastores.

EDIT:

  1. this issue also talks about how to do it and use it as a jumping off point into an external etcd cluster - etcd-io/etcd#10554
@semtexzv

This comment has been minimized.

Copy link

@semtexzv semtexzv commented Sep 30, 2019

Another alternative for HA is something like this, https://github.com/rqlite/rqlite , which is still very lighweight compared to etcd

@liyimeng

This comment has been minimized.

Copy link
Contributor

@liyimeng liyimeng commented Oct 1, 2019

it seems pretty simple to embed etcd, rsqlite is more lightweight in which term? less memory footprint?

@sandys

This comment has been minimized.

Copy link
Author

@sandys sandys commented Oct 1, 2019

@liyimeng thanks for your question.
Well im not claiming sqlite is more lightweight - i think the current default data store of k3s is sqlite because it is embedded inside the k3s daemon (and doesnt need an external etcd service running).

However, etcd can also be embedded - that basically makes it so much simpler for k3s devs since the HA solution is based on etcd only. so they can completely remove the sqlite code.

@sandys sandys mentioned this issue Oct 25, 2019
7 of 7 tasks complete
@cjellick

This comment has been minimized.

Copy link
Member

@cjellick cjellick commented Oct 31, 2019

This isn't really the direction we are taking k3s. We are going to continue to focus on and invest in non-etcd solutions for k3s's database.

For single node installs, the primary motivation for using sqlite is that etcd is much more resource intensive that sqlite. We want this to be able to run well on pretty slim machines.

For HA, check out https://rancher.com/docs/k3s/latest/en/installation/ha/. The recommended HA deployment will be to back k3s with an external SQL database (postgres to start, but mysql soon after). Though, external etcd will also be supported if that is your preference. The primary motivation here is that we believe sql dbs are less challenging and complex to operatre, especially when you consider managed solutions like RDS.

Longer term for HA, we are working on a distributed embedded DB solution. Currently, we are using dqlite for that. See: #300. The motivation for this solution is to make an HA k3s install feel like a simple, single appliance for edge scenarios. Think of managing thousands of small geographically dispersed kubernetes clusters. In this scenario, we don't want the DB to be a separate concern for the operator. We want the operator to have that "all-in-one" experience.

I'm going to close this issue, but we can continue the discussion in comments here if you have more input.

@cjellick cjellick closed this Oct 31, 2019
@sandys

This comment has been minimized.

Copy link
Author

@sandys sandys commented Oct 31, 2019

@cjellick

This comment has been minimized.

Copy link
Member

@cjellick cjellick commented Oct 31, 2019

Makes sense. That's while we'll support both options. Some operators really like the RDS option (or getting a sql db from their in-house db ops team). Others will like the dqlite option (which will have its own set of trade-offs, I think).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.