TestServerCredsDispatch goes into a loop and never passes or fails

[bradfitz]

With both Go 1.7 and Go 1.8 on my machine:

$ go test -v -run=TestServerCredsDispatch google.golang.org/grpc/test
=== RUN   TestServerCredsDispatch
2017/01/26 17:31:12 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::]:40853: connect: network is unreachable"; Reconnecting to {[::]:40853 <nil>}
2017/01/26 17:31:13 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::]:40853: connect: network is unreachable"; Reconnecting to {[::]:40853 <nil>}
2017/01/26 17:31:15 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::]:40853: connect: network is unreachable"; Reconnecting to {[::]:40853 <nil>}
2017/01/26 17:31:18 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::]:40853: connect: network is unreachable"; Reconnecting to {[::]:40853 <nil>}
2017/01/26 17:31:22 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::]:40853: connect: network is unreachable"; Reconnecting to {[::]:40853 <nil>}
...

[abdullah2993]

Same issue over here...

[fengttt]

Also experienced this on 1.8.1, Centos 7.3. Happening one out of 4 or 5 times.

[menghanl]

I'm not able to reproduce this...

I believe the issue behind this is mentioned at golang/go#18806, and it's OS-dependent.
Also, there's a fix for it.

I'm closing this issue now. Please leave your comment here if you think there's some action we need to take.

[bradfitz]

@menghanl, please reopen this. You're relying on undefined behavior.

The Go bug is about defining it, but that would only help Go 1.9+. You need to fix it for earlier Go versions.

The fix is easy: don't listen on ":nnn" and use ln.Addr().String().

[menghanl]

Do you mean: don't listen on ":nnn" and don't use ln.Addr().String()?
And are you suggesting that we should take the approach you mentioned in the golang issue: call TrimPrefix() before dialing?

[bradfitz]

Yes, sorry: you can listen on ":nnn" but if you do so, you're not allowed to use ln.Addr().String() as the dial address, because it's currently undefined.

So you need to clean it up, mapping "0.0.0.0" to "127.0.0.1".

Instead of listening on ":nnn", I recommend using:

func newLocalListener(t *testing.T) net.Listener {
        ln, err := net.Listen("tcp", "127.0.0.1:0")
        if err != nil {
                ln, err = net.Listen("tcp6", "[::1]:0")
        }
        if err != nil {
                t.Fatal(err)
        }
        return ln
}

[menghanl]

Thanks for the suggestion. We will have a fix for this.

[menghanl]

Fix in #1237, PTAL.

One thing to note is that, I'm using "localhost:port", not "127.0.0.1:0" or "[::1]:0".
Please leave a comment if "localhost" doesn't work.

[bradfitz]

In Go's experience, not all machines have "localhost" both defined and defined to be the correct loopback address.

That's why all the Go unit tests use the newLocalListener func I posted above.