os: (*os.File).SetDeadline() doesn't work after io.Copy(*net.TCPConn, *os.File)

[crvv]

What version of Go are you using (go version)?

go version go1.11.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Linux AMD64

What did you do?

This is the same as #22934
io.Copy() uses net.sendFile to do the work, which calls (*os.File).Fd().

go/src/net/sendfile_linux.go

Line 35 in 5ddb209

written, err = poll.SendFile(&c.pfd, int(f.Fd()), remain)

To reproduce, run the following code.

package main

import (
	"fmt"
	"io"
	"net"
	"os"
	"time"
)

func main() {
	r, w, err := os.Pipe()
	if err != nil {
		panic(err)
	}
	go func() {
		_, err := w.Write([]byte("123"))
		if err != nil {
			panic(err)
		}
		time.Sleep(time.Second * 4)
		_, err = w.Write([]byte("456"))
		if err != nil {
			panic(err)
		}
	}()
	go func() {
		l, err := net.Listen("tcp", "localhost:1234")
		if err != nil {
			panic(err)
		}
		defer l.Close()
		conn, err := l.Accept()
		if err != nil {
			panic(err)
		}
		defer conn.Close()
		_, err = io.CopyN(conn, r, 3)
		if err != nil {
			panic(err)
		}
	}()
	go func() {
		time.Sleep(time.Second / 5)
		c, err := net.Dial("tcp", "localhost:1234")
		if err != nil {
			panic(err)
		}
		defer c.Close()
		_, err = io.Copy(os.Stdout, c)
		if err != nil {
			panic(err)
		}
		fmt.Println()
	}()
	time.Sleep(time.Second / 2)

	deadline := time.Now().Add(time.Second)
	err = r.SetDeadline(deadline)
	if err != nil {
		panic(err)
	}
	buffer := make([]byte, 4)
	n, err := r.Read(buffer)
	if err != nil {
		panic(err)
	}
	fmt.Printf("got %v, deadline: %v, now: %v\n", string(buffer[:n]), deadline.Unix(), time.Now().Unix())
}

What did you expect to see?

panic: read |0: i/o timeout

What did you see instead?

got 456, deadline: 1540286817, now: 1540286820

I never encountered this in real applications.
I guess there are more similar issues around Fd() and the runtime poller.
Maybe it's better to just leave this to Go 2 and make Fd not touch the blocking mode.

[flyingtang]

i get this:

123
panic: read |0: i/o timeout

goroutine 1 [running]:
main.main()
        /Users/xg/scan/src/sssss/main.go:66 +0x355exit status 2

[crvv]

@txg5214
I guess you tested this on macOS.
The bug only occurs on platforms where sendfile is used for copying *os.File to *net.TCPConn.
Those are Linux, DragonflyBSD, FreeBSD, Solaris and Windows.

https://github.com/golang/go/blob/master/src/net/sendfile_linux.go
https://github.com/golang/go/blob/master/src/net/sendfile_unix_alt.go
https://github.com/golang/go/blob/master/src/net/sendfile_windows.go

[acln0]

I don't think much can be done here (without adding pointless new API) except to wait for Go 2.

To do sendfile, the net package needs to get at the file descriptor of the os.File, so that it can pass it down into poll.Sendfile. It can't get at its underlying poll.FD, so it has to make due with the integer file descriptor returned by Fd. Calls to Fd have to set the description in blocking mode for historical reasons.

In any case, I doubt this is a problem for any real program. Callers would have to both use a memory mappable descriptor in non-blocking mode, and want to call sendfile on it. And if it really is a problem, it can be worked around using unix.Setnonblock after io.Copy, since the blocking mode is a property of the file description rather than the file descriptor.

[gopherbot]

Change https://golang.org/cl/155137 mentions this issue: os: don't let sendFile put a pipe into blocking mode