grpc-go hangs when used over a synchronous pipe

[madari]

It looks like it is not possible to connect a client to a server over synchronous (in-memory) connection.

What I believe is happening is that both the client and the server attempt to write http/2 control frames/settings (writeOptions etc.) at the same time during the handshake. While they are doing that, neither of them is reading so the writers don't get to make any progress and the handshake gets stuck.

$ go version
go version go1.6.1 darwin/amd64

bug.go (see "BUG" comment):

package main

import (
    "fmt"
    "log"
    "net"
    "time"

    "google.golang.org/grpc"
)

type memAddr string

func (a memAddr) Network() string { return "mem" }
func (a memAddr) String() string  { return string(a) }

type memListener struct {
    c chan net.Conn
}

func (ln *memListener) Accept() (net.Conn, error) {
    conn, ok := <-ln.c
    if !ok {
        return nil, fmt.Errorf("closed")
    }
    return conn, nil
}

func (ln *memListener) Addr() net.Addr {
    return memAddr(fmt.Sprintf("%p", ln))
}

func (ln *memListener) Close() error {
    close(ln.c)
    return nil
}

func main() {
    grpc.EnableTracing = true

    ln := &memListener{
        c: make(chan net.Conn),
    }

    go func() {
        s := grpc.NewServer()
        log.Fatal(s.Serve(ln))
    }()

    log.Printf("Dialing to server over a synchronous pipe...")
    serverConn, err := grpc.Dial("inmemory",
        grpc.WithInsecure(),
        grpc.WithBlock(),
        grpc.WithDialer(func(_ string, _ time.Duration) (net.Conn, error) {
            c1, c2 := net.Pipe()
            log.Printf("Pipe created: %v %v", c1, c2)
            ln.c <- c1
            log.Printf("Pipe accepted: %v %v", c1, c2)
            return c2, nil
        }))
    if err != nil {
        log.Fatal(err)
    }

    // BUG: never reached
    log.Printf("SUCCESS! Connected to server: %v", serverConn)
}

[tamird]

Confirmed that this is fixed by buffering the writes on c1:

diff --git a/main.go b/main.go
index 99c0fdf..7012afa 100644
--- a/main.go
+++ b/main.go
@@ -1,6 +1,7 @@
 package main

 import (
+    "bufio"
     "fmt"
     "log"
     "net"
@@ -35,6 +36,15 @@ func (ln *memListener) Close() error {
     return nil
 }

+type bufferedConn struct {
+    net.Conn
+    *bufio.Writer
+}
+
+func (bc bufferedConn) Write(b []byte) (int, error) {
+    return bc.Writer.Write(b)
+}
+
 func main() {
     grpc.EnableTracing = true

@@ -54,7 +64,7 @@ func main() {
         grpc.WithDialer(func(_ string, _ time.Duration) (net.Conn, error) {
             c1, c2 := net.Pipe()
             log.Printf("Pipe created: %v %v", c1, c2)
-            ln.c <- c1
+            ln.c <- bufferedConn{c1, bufio.NewWriter(c1)}
             log.Printf("Pipe accepted: %v %v", c1, c2)
             return c2, nil
         }))

[iamqizhao]

The issue is that both writes block until the peer completes read. The current code does not work for this kind of synchronous connection. One easy fix is to introduce the ordering for read and write (e.g., server always reads before write) when the connection is setup. But it slows the connection setup process. I am going to think how to resolve this in a decent way.

[iamqizhao]

ok, I got a fix. will send out tomorrow.