Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: (*UDPConn).ReadFrom with zero-byte buffer does not block #23849

Closed
aktungmak opened this issue Feb 15, 2018 · 7 comments

Comments

Projects
None yet
4 participants
@aktungmak
Copy link

commented Feb 15, 2018

What version of Go are you using (go version)?

go1.9.4 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

set GOARCH=amd64
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows

What did you do?

Tried to block on a call to net.PacketConn.ReadFrom

What did you expect to see?

The call to block until data arrives.

What did you see instead?

If I use PacketConn.ReadFrom or UDPConn.ReadFromUDP the call does not block and instead immediately returns 0 bytes and nil error.
It appears that UDPConn.Read() is the only call that actually blocks on Windows.

Extra Info

Relevant previous discussion: https://groups.google.com/forum/#!topic/golang-nuts/mGBeZ5Fphpc
It seems from the link above that this is a known issue (although I could not find any mention here on Github). If a sample program or any other information is needed, please let me know.

@namusyaka namusyaka changed the title udpConn.ReadFrom does not block on windows 7 net: (*UdpConn).ReadFrom does not block on windows 7 Feb 15, 2018

@bradfitz bradfitz changed the title net: (*UdpConn).ReadFrom does not block on windows 7 net: (*UDPConn).ReadFrom does not block on windows 7 Feb 15, 2018

@bradfitz

This comment has been minimized.

Copy link
Member

commented Feb 15, 2018

@bradfitz bradfitz added this to the Go1.11 milestone Feb 15, 2018

@mikioh

This comment has been minimized.

Copy link
Contributor

commented Feb 15, 2018

@aktungmak,

I'm not a Windows user and have no Windows stuff. Also I don't understand what's the real problem with your description: "Tried to block on a call to net.PacketConn.ReadFrom" because I'm not a mind reader. Can you please show us a minimal, repro code? Thanks.

@aktungmak

This comment has been minimized.

Copy link
Author

commented Feb 15, 2018

Hi @mikioh ,

Sure, I will put together some example code to show the issue tomorrow. I have taken a look through the implementation and I will share what I have found so far.

To clarify the expected behaviour, it is my impression that calls like Read, ReadFrom, ReadFromUDP etc should be blocking by default, unless the underlying fd has been modified. This is not what I am seeing though.

Working: UDPConn.Read does block.

Here is the call stack:

conn.Read net/net.go
netFD.Read net/fd_windows.go
pollFD.Read internal/poll/fd_windows.go
syscall.WSARecv syscall/zsyscall_windows.go

r1, _, e1 := Syscall9(procWSARecv.Addr(), 7, uintptr(s), uintptr(unsafe.Pointer(bufs)), uintptr(bufcnt), uintptr(unsafe.Pointer(recvd)), uintptr(unsafe.Pointer(flags)), uintptr(unsafe.Pointer(overlapped)), uintptr(unsafe.Pointer(croutine)), 0, 0)

Not Working: UDPConn.ReadFrom, UDPConn.ReadFromUDP do not block

Here is the call stack,

UDPConn.ReadFrom(UDP)
netFD.readFrom net/fd_windows.go
pollFD.ReadFrom internal/poll/fd_windows.go
syscall.WSARecvFrom syscall/zsyscall_windows.go

r1, _, e1 := Syscall9(procWSARecvFrom.Addr(), 9, uintptr(s), uintptr(unsafe.Pointer(bufs)), uintptr(bufcnt), uintptr(unsafe.Pointer(recvd)), uintptr(unsafe.Pointer(flags)), uintptr(unsafe.Pointer(from)), uintptr(unsafe.Pointer(fromlen)), uintptr(unsafe.Pointer(overlapped)), uintptr(unsafe.Pointer(croutine)))

It seems that WSARecv blocks as expected, but WSARecvFrom does not. Both end up making the same call to Syscall9, but one specifies the source address and the other does not (I don't think that is significant).

My suspicion lies in the file syscall/zsyscall_windows.go, specifically how it sets up the "overlapping" parameter. It seems that overlapping sockets have different blocking behaviour, as described in this document:

https://support.microsoft.com/en-us/help/181611/socket-overlapped-i-o-versus-blocking-nonblocking-mode

The next step is to compare WSARecv and WSARecvFrom to see the differences in how they are set up. I am not familiar with this code so please take this with a pinch of salt.

@mikioh

This comment has been minimized.

Copy link
Contributor

commented Feb 15, 2018

@aktungmak,

With your repro code, please clarify what's meant by your "blocking/non-blocking?" For Go API calls? Windows system service calls? the behavior of Windows transport-layer protocol implementation? If you have a question, the existing test code in the net package might be a help.

@aktungmak

This comment has been minimized.

Copy link
Author

commented Feb 16, 2018

When I say "blocking", I mean that the Go API function (in this case ReadFrom) will not return until there is some data to return (unless there is an error or the socket is closed in another goroutine). I expect that this API would be consistent across platforms.

My understanding of the Read and ReadFrom methods is that they should both have this blocking behavior. However, it seems that ReadFrom returns immediately with zero bytes and nil error on Windows.

Here is the minimal code to reproduce this issue on Windows.

Working, blocks forever (since there is no traffic to receive):


import (
	"log"
	"net"
)

func main() {
	la := &net.UDPAddr{net.IPv4(127, 0, 0, 1), 10101, ""}
	sock, err := net.ListenUDP("udp4", la)
	if err != nil {
		log.Fatalf("can't open listening socket: %s", err)
	}

	buf := []byte{}
	n, err := sock.Read(buf) // blocks until Ctrl-C
	if err != nil {
		log.Fatalf("can't read: %s", err)
	}
	log.Printf("read %d bytes: %v", n, buf)
}

Not working, returns immediately with 0 bytes and nil error:


import (
	"log"
	"net"
)

func main() {
	la := &net.UDPAddr{net.IPv4(127, 0, 0, 1), 10101, ""}
	sock, err := net.ListenUDP("udp4", la)
	if err != nil {
		log.Fatalf("can't open listening socket: %s", err)
	}

	buf := []byte{}
	n, from, err := sock.ReadFrom(buf) // returns immediately!
	if err != nil {
		log.Fatalf("can't read: %s", err)
	}
	log.Printf("read %d bytes from %v: %v", n, from, buf)
}

To compare, I ran the same code on Ubuntu 16.04 and I found that the behaviour is reversed! On linux, sock.Read returns immediately with 0 bytes and nil error, but ReadFrom blocks until it is manually killed with Ctrl-C.

@mikioh

This comment has been minimized.

Copy link
Contributor

commented Feb 16, 2018

buf := []byte{}

If you are asking about the behavior of network IO system calls with "zero-byte user-supplied buffer space", that's not a Go specific question. You may try it with other languages but the most certain way is just to take a look at each operating system kernel's source code; FWIW, the behavior also differs between underlying network- and/or transport-layer protocols.

In short, it depends on each operating system implementation and we now don't try to abstract it because we don't think it's worthy. The test harnesses TestUDPZeroBytePayload and TestUDPZeroByteBuffer in net/udpsock_test.go describe a bit.

Closing. If you want to try to make more abstracted behavior, please open a new issue. Thanks.

@mikioh mikioh closed this Feb 16, 2018

@mikioh mikioh changed the title net: (*UDPConn).ReadFrom does not block on windows 7 net: (*UDPConn).ReadFrom with zero-byte buffer does not block Feb 16, 2018

@aktungmak

This comment has been minimized.

Copy link
Author

commented Feb 16, 2018

Yes, I just realised my mistake here. In my original code I was allocating the buffer like this:

buf := make([]byte, 0, 4096)

and now I realise that it should be like this:

buf := make([]byte, 4096)

This was my mistake so I am happy for it to be closed. Thanks for your support in understanding this!

@golang golang locked and limited conversation to collaborators Feb 16, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.