New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: initial cluster database plumbing #4016

Closed
wants to merge 14 commits into
base: master
from

Conversation

2 participants
@freeekanayaka
Member

freeekanayaka commented Nov 10, 2017

This branch adds the initial plumbing for adding and managing a dqlite database to LXD.

It does NOT introduce a dependency on dqlite yet, but prepares the ground for doing so.

It does however introduce new dependencies on the go-grpc-sql package and all its indirect dependencies, most noticeably the grpc Go package which requires Go 1.8 or higher.

The idea is to first figure out how to have CI happy with this change (for instance we'll have to exclude Go <1.8 from the build) and then gradually move on to include more complicated dependencies like dqlite, which require a custom sqlite build.

@freeekanayaka

This comment has been minimized.

Show comment
Hide comment
@freeekanayaka

freeekanayaka Nov 10, 2017

Member

jenkins: test this please

Member

freeekanayaka commented Nov 10, 2017

jenkins: test this please

@stgraber

This comment has been minimized.

Show comment
Hide comment
@stgraber

stgraber Nov 10, 2017

Member

We won't be able to land this until we have Go 1.9 backported to Ubuntu 16.04 as until then we must keep building with Go 1.6.

Member

stgraber commented Nov 10, 2017

We won't be able to land this until we have Go 1.9 backported to Ubuntu 16.04 as until then we must keep building with Go 1.6.

@stgraber

This comment has been minimized.

Show comment
Hide comment
@stgraber

stgraber Nov 16, 2017

Member

jenkins: test this please

Member

stgraber commented Nov 16, 2017

jenkins: test this please

@stgraber

This comment has been minimized.

Show comment
Hide comment
@stgraber

stgraber Dec 11, 2017

Member

jenkins: test this please

Member

stgraber commented Dec 11, 2017

jenkins: test this please

@stgraber

This comment has been minimized.

Show comment
Hide comment
@stgraber

stgraber Jan 11, 2018

Member

@freeekanayaka do you still benefit from having this one open?

Member

stgraber commented Jan 11, 2018

@freeekanayaka do you still benefit from having this one open?

@freeekanayaka

This comment has been minimized.

Show comment
Hide comment
@freeekanayaka

freeekanayaka Jan 11, 2018

Member

Yes, since we don't have green CI builds/tests for the clustering branch, this PR helps me ensuring that the part of the changeset that does not require dqlite works fine. I guess we can close this as soon as we make CI progress with the clustering branch.

Member

freeekanayaka commented Jan 11, 2018

Yes, since we don't have green CI builds/tests for the clustering branch, this PR helps me ensuring that the part of the changeset that does not require dqlite works fine. I guess we can close this as soon as we make CI progress with the clustering branch.

freeekanayaka added some commits Sep 11, 2017

Add db.Cluster with basic initialization
A new Cluster structure has been added to the lxd/db sub-package. It
is meant to mediate access to the dqlite-based cluster database. It
uses the go-grpc-sql package to serialize SQL queries over a gRPC
connection against the target dqlite leader node.

Signed-off-by: Free Ekanayaka <free.ekanayaka@canonical.com>
Add raft_nodes table
This new table is meant to hold addresses of LXD nodes that are
partecipating to the dqlite raft cluster. Each node in the cluster
will hold its own local copy of this table, regardless of whether it's
a raft node or not.

Signed-off-by: Free Ekanayaka <free.ekanayaka@canonical.com>
Add InsertStrings helper to insert rows with a single string value
Signed-off-by: Free Ekanayaka <free.ekanayaka@canonical.com>
Add db APIs to read and update the raft_nodes table
Signed-off-by: Free Ekanayaka <free.ekanayaka@canonical.com>
Add util.InMemoryNetwork to create in-memory listener/dialer pairs.
This is a convenience for creating in-memory networks that implement
the net.Conn interface. It will be used when running a node in
non-clustered mode, where there will be no actual TCP/gRCP connection
to an external dqlite node, but rather just an in-memory connection to
the local dqlite instance (which will be the leader).

Signed-off-by: Free Ekanayaka <free.ekanayaka@canonical.com>
Add node.DetermineRole function to figure what role a node plays
Signed-off-by: Free Ekanayaka <free.ekanayaka@canonical.com>
Add query helpers to select and insert complex objects
Signed-off-by: Free Ekanayaka <free.ekanayaka@canonical.com>
Rename State.DB to State.Node and add State.Cluster
Signed-off-by: Free Ekanayaka <free.ekanayaka@canonical.com>
Add testing facilities for state.State and sys.OS
Signed-off-by: Free Ekanayaka <free.ekanayaka@canonical.com>
Add V1 cluster schema
This is an initial pass at creating the first version of the cluster
database schema.

An new updateFromV0 patch has been added, which for now only creates a
single table ("nodes") for holding the list of all LXD nodes
participating to the cluster.

Signed-off-by: Free Ekanayaka <free.ekanayaka@canonical.com>
Check the versions of other nodes in cluster.EnsureSchema
Modify cluster.EnsureSchema to also check that all other nodes in the
cluster have a schema version and an API extensions count that match
the ones of the node.

Signed-off-by: Free Ekanayaka <free.ekanayaka@canonical.com>
Add cluster.Gateway to manage the lifecycle of the cluster database
This is a first version of the Gateway object, an API that the daemon
will use in order to 1) run a dqlite node (if appropriate) 2) connect
to the leader dqlite node via gRPC.

For now there's no actual dqlite plumbing in place, and all the
Gateway does is to expose an regular sqlite db over an in-memory gRPC
network (client/server).

Signed-off-by: Free Ekanayaka <free.ekanayaka@canonical.com>
Wire cluster.Gateway into Daemon
Signed-off-by: Free Ekanayaka <free.ekanayaka@canonical.com>
Wire cluster.EnsureSchema into db.OpenCluster
Signed-off-by: Free Ekanayaka <free.ekanayaka@canonical.com>
@freeekanayaka

This comment has been minimized.

Show comment
Hide comment
@freeekanayaka

freeekanayaka Jan 19, 2018

Member

Closing this PR since #4094 is basically green (albeit with clustering tests temporarily disabled).

Member

freeekanayaka commented Jan 19, 2018

Closing this PR since #4094 is basically green (albeit with clustering tests temporarily disabled).

@freeekanayaka freeekanayaka deleted the freeekanayaka:initial-cluster-database-plumbing branch Jan 19, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment