Schema tracking in vtgate #8074
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
systay
added
Component: Query Serving
Type: Enhancement
systay
force-pushed
the
schema-tracking-vtgate
branch
from
5c1c251
to
9f7a7ff
Compare
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
harshit-gangal
force-pushed
the
schema-tracking-vtgate
branch
from
8e6f5a3
to
410cd4c
Compare
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
harshit-gangal
force-pushed
the
schema-tracking-vtgate
branch
from
36a4841
to
31294da
Compare
…vtopo.Server directly Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
…rom schema tracker Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
…ema.Tracker Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Co-authored-by: Andres Taylor <andres@planetscale.com> Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
…chema when errors Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
systay
force-pushed
the
schema-tracking-vtgate
branch
from
0c52ebe
to
3c11a1c
Compare
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
systay
force-pushed
the
schema-tracking-vtgate
branch
from
7cada71
to
369ad4a
Compare
Co-authored-by: Andres Taylor <andres@planetscale.com> Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Co-authored-by: Andres Taylor <andres@planetscale.com> Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
…tgate Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
harshit-gangal
requested review from
deepthi,
GuptaManan100,
shlomi-noach and
systay
as code owners
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
2 tasks
harshit-gangal
commented
Jun 1, 2021
Comment on lines
+106
to
+114
| clusterInstance.VtTabletExtraArgs = []string{"-queryserver-config-schema-change-signal"} | ||
| err = clusterInstance.StartUnshardedKeyspace(*keyspace, 1, false) | ||
| require.NoError(t, err) | ||
|
|
||
| // teardown vttablets | ||
| for _, vttablet := range clusterInstance.Keyspaces[0].Shards[0].Vttablets { | ||
| err = vttablet.VttabletProcess.TearDown() | ||
| require.NoError(t, err) | ||
| } |
There was a problem hiding this comment.
instead of starting tablets and shutting them down, we can test this by just creating the keyspace and shard in topo by calling vtctld APIs (CreateKeyspace and CreateShard). That will mimic the same behavior.
go/test/endtoend/vtgate/schematracker/loadkeyspace/schema_load_keyspace_test.go
Show resolved
Hide resolved
go/test/endtoend/vtgate/schematracker/restarttablet/schema_restart_test.go
Show resolved
Hide resolved
Comment on lines
+161
to
+165
| if v == nil { | ||
| // We encountered an error, we should always have a current vschema | ||
| log.Warning("got a schema changed signal with no loaded vschema. if this persist, something is wrong") | ||
| vschema, _ = vindexes.BuildVSchema(&vschemapb.SrvVSchema{}) | ||
| } else { |
There was a problem hiding this comment.
if currentSrvSchema is nil we do not need to rebuild the VSchema as the empty VSchema would have been published to the subscriber through VSchemaUpdate
This should only be done in the case when we are using updated schema to enhance the VSchema
Comment on lines
+1160
to
+1162
| if vschema != nil { | ||
| e.vschema = vschema | ||
| } |
There was a problem hiding this comment.
This can create an issue when a user wants to explicitly clear out the VSchema.
This check will ignore the update and will keep the old version.
systay
approved these changes
Jun 1, 2021
This was referenced Jun 22, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This adds a schema tracker that will update the vschema with schema information about tables and columns, fetched from the underlying MySQL instance.
Related Issue(s)
#7995
#7994
Checklist