net: malformed payload of UDP packets on Mac

[bartoszmodelski]

Go version: go1.10.3 darwin/amd64
Does this issue reproduce with the latest release? Yes
OS and architecture: darwin/amd64 (OS High Sierra 10.13.3)

What did you do?

Listened to SIP communciation on UDP.

Code: https://play.golang.org/p/fqTQEMb70Is (naturally, needs to be run locally)

What did you expect to see?

Correct SIP Invite packet arriving every second, or so.

First 16 characters of a correct SIP Invite:
00000000 49 4e 56 49 54 45 20 73 69 70 3a 31 34 35 36 40 |INVITE sip:1456@|

What did you see instead?

First 4 character of some packets are malformed. Namely, the "INVI" was overwritten with two CRLF.

Initially, about 30% of the packets have the issue but the longer the server executes, the higher part of the whole they constitute. First 16 characters of a malformed packet:
00000000 0d 0a 0d 0a 54 45 20 73 69 70 3a 31 34 35 36 40 |....TE sip:1456@|

• Rests of the packets look good.
• Wireshark confirmed, that the anomaly is not on sender's side.
• The double CRLF is always present later in the packet at position 239 (hex).
• This behavior occurs when using Read/ReadFrom/ReadFromUDP
• Example of sent data: https://pastebin.com/bzguwfYY
• Doesn't happen when a new buffer is allocated for every call to Read(). (That gives me some empty packets, but does not look like any of the actually send messages has been lost.)

[josharian]

cc @mikioh

[mikioh]

You need to deal with the number of bytes received appropriately like the following:

b := make([]byte, 2048)
for {
	n, err := c.Read(b)
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Printf("%s", b[:n])

For more questions, please take a look at https://github.com/golang/go/wiki/Questions.

[as]

@mikioh example clarification: Shouldn't that print come before the error check?

b := make([]byte, 2048)
for {
	n, err := c.Read(b)
	fmt.Printf("%s", b[:n])
	if err != nil {
		fmt.Println(err)
		return
	}

go doc io.Reader

Callers should always process the n > 0 bytes returned before considering
the error err. Doing so correctly handles I/O errors that happen after
reading some bytes and also both of the allowed EOF behaviors.

[mikioh]

Shouldn't that print come before the error check?

The net.UDPConn conforms to not only io.Reader interface but net.Conn and net.PacketConn interfaces. In addition, the net.UDPConn doesn't implement a few features that satisfy io.Reader interface such as returning io.EOF at the message, I mean, a unit of data exchange, boundary.

Can we rely on io.Reader interface over net.UDPConn? That's a good question, though, I still have no concrete answer. It's probably okay w/ a few limitations such as io.EOF on almost all the existing implementations, but there will probably be exceptions, for example, on platforms enabling fancy LRO (large-receive-optimization) under heavy load, a microcode in some networking peripheral may advise that it perhaps returns an error with a corrupted number of bytes transferred because of the lack of spare logic-gates or machine cycles.