have trouble to use D grpc client to talk to Python grpc server

[mw66]

Hi,

I'm trying the helloworld example, but I'm using a Python server, something like this:

import grpc
import HW_pb2
import HW_pb2_grpc

if __name__ == "__main__":
  handler = HWHandler()
  server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
  HW_pb2_grpc.add_HWServicer_to_server(handler, server)
  server.add_insecure_port('[::]:' + str(PORT))
  server.start()

And then use the D grpc client talk to it, the 1st time I run it: I got:

Thread 11 "" received signal SIGSEGV, Segmentation fault.                
[Switching to Thread 0x7fffcf7fe700 (LWP 2892)]                                                                
0x0000555555aca53c in _D4hunt4http6client18Http1ClientDecoderQu6decodeMFCQBx2io10ByteBufferQmCQCs3net10ConnectionQmZv (warning: (Internal error: pc 0x555
555a69009 in read in psymtab, but not in symtab.)                                  
                                                                                                                                                        
warning: (Internal error: pc 0x555555a68fa0 in read in psymtab, but not in symtab.)
                                                                                   
warning: (Internal error: pc 0x555555a69009 in read in psymtab, but not in symtab.)                                                                      
                                                                                                                                                         
warning: (Internal error: pc 0x555555a69009 in read in psymtab, but not in symtab.)
                                                                                                                                                         
this=0x7fffec8589c0, warning: (Internal error: pc 0x555555a69009 in read in psymtab, but not in symtab.)
                                                                                                                                                         
                                                                                                                                       
buffer=0x7fffeccd3e40, warning: (Internal error: pc 0x555555a69009 in read in psymtab, but not in symtab.)                                               
                                                                                   
session=0x7fffecd36690) at Http1ClientDecoder.d:44                                                                           
44                  throw new IllegalStateException("Client connection is null! The actual type is: "                

The 2nd time:

grpc.GrpcException.GrpcTimeoutException@/home//.dub/packages/grpc-0.3.0-beta.4/grpc/source/grpc/GrpcStream.d(225): Timedout after 5 seconds.
----------------                                                                      
??:? [0x555555d64775]
??:? [0x555555d8c9f6]
??:? [0x555555d7056d]
/home//.dub/packages/grpc-0.3.0-beta.4/grpc/source/grpc/GrpcStream.d:225 [0x55555574a6cb]

Since it's a very simple grpc server, before I investigate further, I want to ask: does this D library's grpc client can talk to a Python grpc server?

The Python side grpc code are generated with:

python -m grpc_tools.protoc -I...

with vesion:

grpcio                             1.31.0    
grpcio-tools                       1.31.0    
protobuf                           3.13.0    

If you know these 2 can work together, can you provide a Python grpc server example? and with the specific Python package version?

Thanks.

[Heromyth]

It seems the handshake failed with Python server.
It's so sad that we have not tested this with Python.

[mw66]

@Heromyth Thanks for the confirmation.

Hi, I created this PR: Python server & client for test with D client & server

#16

Please try it.

Thanks.

[mw66]

Test D client with Python server:

$ make pyproto

then test the Python server:

$ cd source
$ python3 server.py
$ python3 client.py  # in another terminal
message: "hi, Hunt"

Now test D client

$ ./client

[Heromyth]

Thanks for your PR. We will test it to find out what happened.

[mw66]

The other way does not work either:

$ SimpleDemo$ ./server   # D server, can see the request
2020-Nov-25 20:21:23.0448696 | 70846 | trace | GreeterImpl.SayHello | request: Hunt, reply: Hello Hunt | source/GreeterImpl.d:15

$ source$ python3 client.py   # python client in another terminal, but didn't get response.

[Heromyth]

We are testing this. However, nothing has been found.

[TimCoraxAudio]

We are testing this. However, nothing has been found.

Are able to let s know what software you are using. If there are differences, that might highlight an issue. It looks like it could be a mismatched socket interface

[Heromyth]

Here is the example: https://github.com/huntlabs/grpc-dlang/tree/master/examples/SimpleDemo.
It's OK for the server and client in D. There are problems for the server in D and the client, or the converse.
We are using a tool called Wireshark to monitor the communication, try to find out the difference.

By the way, adding some debug switchers to dub.json, you can get more logs for debug.

	"versions": [
		"HUNT_DEBUG", "HUNT_HTTP_DEBUG", "HUNT_NET_DEBUG"
	],

[mw66]

@Heromyth

Do you want to add this issue to GSOC (Google Summer of Code) projects? so someone can try to fix the network issue, and be awarded by Google?

https://github.com/dlang/projects/issues?q=is%3Aissue+is%3Aopen+label%3Agsoc2021

[Imperatorn]

We need to fix this

[kubo39]

#22 and #23 will fix the server side problem.
D grpc client still has some problems, I'll investigate.

[Heromyth]

Thansk. All the patches are merged.

[kubo39]

The client cannot finish successfully.
It seems that the problem is there's no way to stop hunt.net.NetUtil.eventLoop explicitly.
Ah, I'm wrong. This uses EventLoopPool internally.

[kubo39]

Tried,

    // Disconnect.
    channel.destroy();
    import hunt.http.Util;
    CommonUtil.stopScheduler;
    import hunt.net.EventLoopPool;
    shutdownEventLoopPool;

but the client didn't stop.

[Heromyth]

Tried,

    // Disconnect.
    channel.destroy();
    import hunt.http.Util;
    CommonUtil.stopScheduler;
    import hunt.net.EventLoopPool;
    shutdownEventLoopPool;

but the client didn't stop.

You can use gdb to debug this, and check which thread is stopping the client from exit.

[mw66]

Just an update on the progress (for the new comers): right now Python client can talk to D server, thanks @kubo39 ; but the other way still does not work. People can give the demo a try:

https://github.com/huntlabs/grpc-dlang/tree/master/examples/SimpleDemo

Since D's strength is on the server side, I'd say 60% of the bug fixed :-)