net/http: ptrace protections result in hung sendfile test

[daviddengcn]

On this revision:

commit 886b02d705ffb1be8b4974ac4c355d480a24e3ec
Author: Andrew Gerrand <adg@golang.org>
Date:   Thu Jan 15 15:21:14 2015 +1100

    [release-branch.go1.4] go1.4.1

    Change-Id: If275a5caa07cfd16b7052ad50709e1d0f1258223
    Reviewed-on: https://go-review.googlesource.com/2856
    Reviewed-by: Rob Pike <r@golang.org>

Run all.bash, one of the testcases failed:

panic: test timed out after 2m0s

goroutine 264 [running]:
testing.func·008()
    /home/david/go-git/src/testing/testing.go:681 +0x12f
created by time.goFunc
    /home/david/go-git/src/time/sleep.go:129 +0x4b

goroutine 1 [chan receive]:
testing.RunTests(0x8fcff8, 0xa371e0, 0xd2, 0xd2, 0xc20804c001)
    /home/david/go-git/src/testing/testing.go:556 +0xad6
testing.(*M).Run(0xc20804c0f0, 0x0)
    /home/david/go-git/src/testing/testing.go:485 +0x6c
net/http_test.TestMain(0xc20804c0f0)
    /home/david/go-git/src/net/http/main_test.go:19 +0x28
main.main()
    net/http/_test/_testmain.go:510 +0x1d1

goroutine 262 [IO wait]:
net.(*pollDesc).Wait(0xc20821d790, 0x72, 0x0, 0x0)
    /home/david/go-git/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20821d790, 0x0, 0x0)
    /home/david/go-git/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20821d730, 0xc2080c2000, 0x1000, 0x1000, 0x0, 0x7ff8d880c070, 0xc20819e4b8)
    /home/david/go-git/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208032248, 0xc2080c2000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /home/david/go-git/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7ff8d880f2e0, 0xc208032248, 0xc20817a7e8, 0xc2080c2000, 0x1000, 0x1000, 0x7ff8d87fa010, 0x0, 0x0)
    /home/david/go-git/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc208181f80, 0xc2080c2000, 0x1000, 0x1000, 0xc207ffcdda, 0x0, 0x0)
    <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080a1980)
    /home/david/go-git/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080a1980, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /home/david/go-git/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20817a790)
    /home/david/go-git/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
    /home/david/go-git/src/net/http/transport.go:660 +0xc9f

goroutine 259 [select]:
net/http.(*persistConn).roundTrip(0xc20817a790, 0xc20819e420, 0x0, 0x0, 0x0)
    /home/david/go-git/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc208076000, 0xc2081fd1e0, 0xc208181e60, 0x0, 0x0)
    /home/david/go-git/src/net/http/transport.go:235 +0x558
net/http.send(0xc2081fd1e0, 0x7ff8d880c468, 0xc208076000, 0x17, 0x0, 0x0)
    /home/david/go-git/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xa395a0, 0xc2081fd1e0, 0x17, 0x0, 0x0)
    /home/david/go-git/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xa395a0, 0xc2081fd1e0, 0x8fd318, 0x0, 0x0, 0x0)
    /home/david/go-git/src/net/http/client.go:367 +0xb25
net/http.(*Client).Get(0xa395a0, 0xc208181e00, 0x17, 0xc2081fc0d0, 0x0, 0x0)
    /home/david/go-git/src/net/http/client.go:299 +0xba
net/http.Get(0xc208181e00, 0x17, 0xc20805be58, 0x0, 0x0)
    /home/david/go-git/src/net/http/client.go:276 +0x50
net/http_test.TestLinuxSendfile(0xc2080774d0)
    /home/david/go-git/src/net/http/fs_test.go:822 +0xb14
testing.tRunner(0xc2080774d0, 0xa378a0)
    /home/david/go-git/src/testing/testing.go:447 +0xbf
created by testing.RunTests
    /home/david/go-git/src/testing/testing.go:555 +0xa8b

goroutine 263 [select]:
net/http.(*persistConn).writeLoop(0xc20817a790)
    /home/david/go-git/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
    /home/david/go-git/src/net/http/transport.go:661 +0xcbc
FAIL    net/http    120.018s

[ianlancetaylor]

cc @bradfitz

[adg]

It would help to know more about your system. What OS? What hardware?

[daviddengcn]

It is a linode host. Hope this helps:

$ uname -a
Linux li69-153 3.15.3-x86_64-linode44 #1 SMP Tue Jul 1 15:51:04 EDT 2014 x86_64 GNU/Linux
$ gcc --version
gcc (Ubuntu 4.4.3-4ubuntu5.1) 4.4.3

Glad to provide more information that helps. Just tell me the linux command to run.

[adg]

Does it consistently fail? Is the machine really low specced? If the machine is just very slow, then tests can timeout sometimes.

[daviddengcn]

It consistently failed on my machine. I just killed all the background processes that may consume resource and ran it again. The testcase was still timeout. All other cases were finished within 2s.

[alexcesaro]

I also got this test failing on a Docker container (Ubuntu 12.04) while the test pass outside the container. It fails on tags go1.4 and go1.4.2.

http.Get timeouts here: https://github.com/golang/go/blob/release-branch.go1.4/src/net/http/fs_test.go#L822

[alexcesaro]

I just had a look and the test fails because strace returns an error: strace: ptrace(PTRACE_TRACEME, ...): Permission denied

It happens because of a ptrace protection: https://wiki.ubuntu.com/SecurityTeam/Roadmap/KernelHardening#ptrace_Protection

Since strace is run asynchronously, the test does not see the error and http.Get timeouts. So there is two ways to fix that issue:

• checking the content of /proc/sys/kernel/yama/ptrace_scope
• using child.Run() in a goroutine instead of child.Start()

I think the second fix is the best one. I can send a CL if needed.