Skip to content
This repository has been archived by the owner on Dec 16, 2022. It is now read-only.

[vtctld] Custom build for vtctld development #195

Open
wants to merge 21 commits into
base: slack-vitess-8-2021.03.03r0.wip
Choose a base branch
from

Conversation

setassociative
Copy link

@setassociative setassociative commented Mar 5, 2021

Branch construction -- cherry-pick results (x - merged fine, c - conflict)

rbailey@localhost $ git rev-list --reverse tinyspeck/master...slack-vitess-2021.01.11-release --format="%h - %as - %ae - %s" | grep -v commit
x - 2f2635c9d - 2020-12-10 - deepthi@planetscale.com - Merge pull request #7128 from tinyspeck/am_vtctld_proto
x - 6b7bb43c8 - 2021-01-10 - amason@slack-corp.com - Commit go.sum change after running `make build`
x - bb80e80ef - 2020-12-23 - 57520317+rohit-nayak-ps@users.noreply.github.com - Merge pull request #7201 from tinyspeck/am_find_all_shards_in_keyspace
x - bb3c5c12b - 2021-01-05 - deepthi@planetscale.com - Merge pull request #7238 from tinyspeck/am_vtctldclient_generator
x - 86c518c9b - 2021-01-11 - rafael@slack-corp.com - Merge pull request #7285 from tinyspeck/am_fix_findallshardsinkeyspace
x - 726c30663 - 2021-01-15 - 57520317+rohit-nayak-ps@users.noreply.github.com - Merge pull request #7302 from tinyspeck/am_vtctld_cell_getters
s - a874ff75f - 2021-02-11 - amason@slack-corp.com - Commit go.mod changes after running `make build`
x - b29d8198f - 2021-01-18 - 57520317+rohit-nayak-ps@users.noreply.github.com - Merge pull request #7311 from tinyspeck/am_vtctld_tablet_getters
x - 3673c6df8 - 2021-01-19 - deepthi@planetscale.com - Merge pull request #7321 from tinyspeck/am_vtctldclient_command_pkg
x - 75dca95af - 2021-01-21 - 57520317+rohit-nayak-ps@users.noreply.github.com - Merge pull request #7334 from tinyspeck/am_vtctld_getsrvvschema
c - 70fe75118 - 2021-01-22 - 57520317+rohit-nayak-ps@users.noreply.github.com - Merge pull request #7346 from tinyspeck/am_vtctld_getschema
    - deleted endtoend test for GetSchema
    - vtctl.go -- updated commandGetSchema return, rm tableSizesOnly flag in proto flag doesn't exist in branch
x - 5dbea28ad - 2021-01-23 - 57520317+rohit-nayak-ps@users.noreply.github.com - Merge pull request #7352 from tinyspeck/am_vtctld_listbackups
x - a9d62846b - 2021-01-24 - 57520317+rohit-nayak-ps@users.noreply.github.com - Merge pull request #7360 from tinyspeck/am_vtctld_getvschema
c - 0082c4d09 - 2021-02-04 - 57520317+rohit-nayak-ps@users.noreply.github.com - Merge pull request #7395 from tinyspeck/am_vtctld_manage_topo
    - function comment
x - d71cf9c25 - 2021-02-02 - deepthi@planetscale.com - Merge pull request #7404 from tinyspeck/am_vtctldclient_silence_errors
x - 6c21469aa - 2021-02-05 - 57520317+rohit-nayak-ps@users.noreply.github.com - Merge pull request #7451 from tinyspeck/am_vtctldclient_errors
x - bea6aeeb8 - 2021-02-11 - amason@slack-corp.com - Apply missing test initializer fix for grpcvtctldclient
c - 71d04f289 - 2021-02-21 - deepthi@planetscale.com - Merge pull request #7518 from tinyspeck/am_testtmclient_refactor
    - deleted endtoend test for GetSchema
x - fcaf444b4 - 2021-03-04 - 57520317+rohit-nayak-ps@users.noreply.github.com - Merge pull request #7575 from tinyspeck/am_vtctld_workflows
x - cd78f4e7e - 2021-03-05 - amason@slack-corp.com - Backport GetWorkflows to use old sqlparser types in cherry-pick branch

deepthi and others added 20 commits March 11, 2021 15:11
Initial implementation of vtctld service
Signed-off-by: Andrew Mason <amason@slack-corp.com>
…_keyspace

Add FindAllShardsInKeyspace to vtctldserver
…ator

[vtctldclient] vtctldclient generator
…nkeyspace

[vtctld] Fix accidentally-broken legacy vtctl output format
[vtctld] Migrate GetSchema

Signed-off-by: Richard Bailey <rbailey@slack-corp.com>
[vtctld] Migrate ListBackups as GetBackups in new vtctld server
[vtctld] Migrate GetVSchema to VtctldServer
[vtctld] Migrate topo management RPCs

Signed-off-by: Richard Bailey <rbailey@slack-corp.com>
…ce_errors

[vtctldclient] Set `SilenceErrors` on the root command, so we don't double-log
Provide named function for squashing usage errors; start using it
Signed-off-by: Andrew Mason <amason@slack-corp.com>
[vtctld | tests only] testtmclient refactor
Conflicts:
	go/vt/vtadmin/api_test.go
	go/vt/vtctl/reparentutil/emergency_reparenter_test.go
	go/vt/vtctl/reparentutil/planned_reparenter_test.go

Resolved by `git rm -f` those files.

Note: Also includes a change to grpcvtctldclient/client_test.go, not in the
cherry-picked commit, to use the testutil constructor for vtctldservers.

Signed-off-by: Andrew Mason <amason@slack-corp.com>

wip test fixes

Signed-off-by: Andrew Mason <amason@slack-corp.com>

Signed-off-by: Richard Bailey <rbailey@slack-corp.com>
[vtctld] Add v0 GetWorkflows rpc and workflow/vexec packages

Signed-off-by: Andrew Mason <amason@slack-corp.com>
Conflicts:
      go/vt/proto/vtctldata/vtctldata.pb.go
      go/vt/proto/vtctlservice/vtctlservice.pb.go
      go/vt/vtctl/grpcvtctldclient/client_gen.go
      go/vt/vtctl/grpcvtctldserver/server.go
      go/vt/wrangler/vexec.go
      proto/vtctldata.proto
      proto/vtctlservice.proto
Signed-off-by: Andrew Mason <amason@slack-corp.com>
Signed-off-by: Richard Bailey <rbailey@slack-corp.com>
@setassociative setassociative force-pushed the slack-vitess-8-2021.03.03r0.vtctld branch from 33ffe0b to 1f6f322 Compare March 11, 2021 23:16
Copy link

@ajm188 ajm188 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks good to me!


package main

// Imports and register the gRPC vtctld client.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ugh i will fix this wording upstream

@@ -1383,9 +1383,6 @@ func (m *VEvent) GetJournal() *Journal {

func (m *VEvent) GetDml() string {
if m != nil {
if m.Statement != "" {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just calling out this diff because github hides it; a long time ago someone manually edited this proto file to include this check, which running make proto removes. Upstream was fine with this change.

@@ -1773,20 +1760,29 @@ func commandGetKeyspace(ctx context.Context, wr *wrangler.Wrangler, subFlags *fl
}

keyspace := subFlags.Arg(0)
keyspaceInfo, err := wr.TopoServer().GetKeyspace(ctx, keyspace)

keyspaceInfo, err := wr.VtctldServer().GetKeyspace(ctx, &vtctldatapb.GetKeyspaceRequest{
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calling this out as a code change in the legacy server, which is backwards compatible, but technically a change

}

func commandGetKeyspaces(ctx context.Context, wr *wrangler.Wrangler, subFlags *flag.FlagSet, args []string) error {
keyspaces, err := wr.TopoServer().GetKeyspaces(ctx)
resp, err := wr.VtctldServer().GetKeyspaces(ctx, &vtctldatapb.GetKeyspacesRequest{})
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calling this out as a code change in the legacy server, which is backwards compatible, but technically a change

@@ -2224,11 +2220,21 @@ func commandFindAllShardsInKeyspace(ctx context.Context, wr *wrangler.Wrangler,
}

keyspace := subFlags.Arg(0)
result, err := wr.TopoServer().FindAllShardsInKeyspace(ctx, keyspace)
result, err := wr.VtctldServer().FindAllShardsInKeyspace(ctx, &vtctldatapb.FindAllShardsInKeyspaceRequest{
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calling this out as a code change in the legacy server, which is backwards compatible, but technically a change

@@ -2311,17 +2317,25 @@ func commandGetSchema(ctx context.Context, wr *wrangler.Wrangler, subFlags *flag
excludeTableArray = strings.Split(*excludeTables, ",")
}

sd, err := wr.GetSchema(ctx, tabletAlias, tableArray, excludeTableArray, *includeViews)
resp, err := wr.VtctldServer().GetSchema(ctx, &vtctldatapb.GetSchemaRequest{
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calling this out as a code change in the legacy server, which is backwards compatible, but technically a change

@@ -28,3 +28,70 @@ import "vtctldata.proto";
service Vtctl {
rpc ExecuteVtctlCommand (vtctldata.ExecuteVtctlCommandRequest) returns (stream vtctldata.ExecuteVtctlCommandResponse) {};
}

// Service Vtctld exposes gRPC endpoints for each vt command.
service Vtctld {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confirming this has everything needed by vtadmin

* Merge pull request #187 from tinyspeck/backport-am_get_schema_sizes_only

Add option to GetSchema to only send the row count and data length over the wire

* whitespace!

* derp, actually commit rm dead import

Signed-off-by: Richard Bailey <rbailey@slack-corp.com>

Co-authored-by: Andrew Mason <amason@slack-corp.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
5 participants