vtgate go low level client reworked

[sougou]

I've written tests (93% coverage) that use the new framework.
We now have a separate VTGateTx interface separate from VTGateConn. So, multiplexing on VTGateConn is now possible.

[alainjobart]

Overall, it looks good from the client API point of view.
From the low level implementation though, each protocol will have to implement the session management. I think I'd rather have the low level RPC connector just expose the RPC API (that includes session), while the higher level client API is shared by all RPC layers (and has the API you defined).

We can do that refactoring later though, this is good enough for the people who want it ASAP.

[sougou]

That was what I originally considered. But then, the difference between that vs. the current one was barely 10 lines (the two Execute functions in Tx).
It felt like a decent trade-off because we avoid another layer. But we'll see how it evolves.

[alainjobart]

The testing part ends up harder too then. You can't unit test the RPC layer without also having to test the Session code. So the tests end up being bigger than what they need to be (could have a unit test with a stubbed client RPC layer that tests the session code layer and be done with that part, and then each RPC implementation only tests the RPC).

But as I said, I'm fine going with this for now.

[mitiger]

@sougou

why the function dial in gorpcvtgateconn/conn.go is private ?

[sougou]

You have to dial using this package: https://github.com/youtube/vitess/blob/master/go/vt/vtgate/vtgateconn/vtgateconn.go. We have the additional layer of abstraction so we can support other RPC protocols.

Or, you can use the go driver as soon as #465 gets merged.

[mitiger]

is the client finished?
I will try to use it right now

[sougou]

Yes. You can use the tests as example for now.

[mitiger]

@sougou
how to use the client ?
shoule i implement the func DialerFunc myself?

func TestRegisterDialer(t *testing.T) {
dialerFunc := func(context.Context, string, time.Duration) (VTGateConn, error) {
return nil, nil
}
RegisterDialer("test1", dialerFunc)
RegisterDialer("test1", dialerFunc)
}

in the test function, return two nil ???

[sougou]

You can start with these tests: https://github.com/youtube/vitess/blob/master/go/vt/client/client_test.go.
In the tests, we create a fake VTGate server, but you should be able to adopt them to use a real VTGate.

[mitiger]

how can i tell vtgate client which keyspace to use?

in python client :
cursor = conn.cursor('kindc', 'master', keyranges=UNSHARDED, writable=True)
here , it tell keyspace is 'kindc' which is my keyspace.

but in golang's client ,i don't know , how to use?
the error info as follow:
cannot route query: insert into test_table(msg) values (?): table test_table not found, vtgate: http://PowerEdge-R410-106:15001/

but , the table is in kindc, and use python's client insert data is ok!!

[sougou]

Ahh. The go client uses the V3 API, where you don't have specify kespace or keyspace id. However, you have to push a 'vschema' config into the topo that contains information about how to route the queries. I have a design doc here: https://github.com/youtube/vitess/blob/master/doc/VTGateV3.md.
The python example to look at is here: https://github.com/youtube/vitess/blob/master/test/vtgatev3_test.py
This thread has some good examples also: https://groups.google.com/forum/?pli=1#!topic/vitess/EcUB0mIrLN4

[mitiger]

@sougou

if i need to rebuild vtgate , and restart vtgate process?
The new vtgate will auto find my test_table in which keyspace?

[mitiger]

@sougou
sorry , i can not understand，what should i need to do in my application?
i need create the json string like :
schema = '''{
"Keyspaces": {
"user": {
"Sharded": true,
"Vindexes": {
"user_index": {
"Type": "hash_autoinc",
"Params": {
"Table": "vt_user_idx",
"Column": "id"
},
"Owner": "vt_user"
}
........

or need create a topo.Keyspace instance? but if i create the instance , how can i set the instance to my connection that connected to the vtgate?

can you give me a complete example that can show me how to use the golang's client?

[mitiger]

@sougou
i know something about the json file . it used in vtgate start command.

now , i can use the vtgate, and select some data from my test_table

but when i insert data , an error appeared.
exec error: shard, host: kindc.0.master, {Uid:1 Host:PowerEdge-R410-106 NamedPortMap:map[mysql:3307 vt:14001 _vtocc:14001] Health:map[]}, vttablet: not_in_tx: DMLs not allowed outside of transactions

this is my vtgate json:
{
"Keyspaces": {
"kindc": {
"Tables": {
"test_table": "test_table"
},
"Sharded": false,
"Classes": {
"test_table": {
"ColVindexes": [
{
"Col": "id",
"Name": "test_table_index"
}
]
}
},
"Vindexes": {
"test_table_index": {
"Params": {
"Table": "test_table_idx",
"Column": "id"
},
"Type": "numeric",
"Owner": "test_table"
}
},
"lookup":{
"Sharded":false,
"Tables":{
"test_table_idx":""
}
}
}
}
}

something wrong for my json file?
and can you give me a document for the json , what about the json's key mean ? in VTGateV3.md , the explain about the json file is not detailed.

thank you very much .

[sougou]

The best way to verify the correctness of the json (or create a new one) is to load it into the schema editor tool in vtctld.
But if vtgate loaded the vschema successfully, that also means that the schema must be good. The particular error you're seeing is basically saying that you're sending a DML without starting a transaction. Did you remember to call Begin? Can you show us your code?

[mitiger]

i know .
thank you , i will try again a few later.

[mitiger]

@sougou
i am very glad , because my demo run success.
thank you very mach!!!

[sougou]

Congratulations! I'm impressed :).
Let us know if you need more help.