Skip to content

/go

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: ptrace protections result in hung sendfile test #9711

Closed
daviddengcn opened this issue Jan 28, 2015 · 11 comments

Comments

Assignees
No one assigned
Labels
FrozenDueToAge Testing help wanted
Projects
None yet
Milestone
  Go1.9
8 participants
@daviddengcn @ianlancetaylor @adg @alexcesaro @bradfitz @kennygrant @gopherbot @rsc
@daviddengcn
Copy link

commented Jan 28, 2015

On this revision:

commit 886b02d705ffb1be8b4974ac4c355d480a24e3ec
Author: Andrew Gerrand <adg@golang.org>
Date:   Thu Jan 15 15:21:14 2015 +1100

    [release-branch.go1.4] go1.4.1

    Change-Id: If275a5caa07cfd16b7052ad50709e1d0f1258223
    Reviewed-on: https://go-review.googlesource.com/2856
    Reviewed-by: Rob Pike <r@golang.org>

Run all.bash, one of the testcases failed:

panic: test timed out after 2m0s

goroutine 264 [running]:
testing.func·008()
    /home/david/go-git/src/testing/testing.go:681 +0x12f
created by time.goFunc
    /home/david/go-git/src/time/sleep.go:129 +0x4b

goroutine 1 [chan receive]:
testing.RunTests(0x8fcff8, 0xa371e0, 0xd2, 0xd2, 0xc20804c001)
    /home/david/go-git/src/testing/testing.go:556 +0xad6
testing.(*M).Run(0xc20804c0f0, 0x0)
    /home/david/go-git/src/testing/testing.go:485 +0x6c
net/http_test.TestMain(0xc20804c0f0)
    /home/david/go-git/src/net/http/main_test.go:19 +0x28
main.main()
    net/http/_test/_testmain.go:510 +0x1d1

goroutine 262 [IO wait]:
net.(*pollDesc).Wait(0xc20821d790, 0x72, 0x0, 0x0)
    /home/david/go-git/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20821d790, 0x0, 0x0)
    /home/david/go-git/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20821d730, 0xc2080c2000, 0x1000, 0x1000, 0x0, 0x7ff8d880c070, 0xc20819e4b8)
    /home/david/go-git/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208032248, 0xc2080c2000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /home/david/go-git/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7ff8d880f2e0, 0xc208032248, 0xc20817a7e8, 0xc2080c2000, 0x1000, 0x1000, 0x7ff8d87fa010, 0x0, 0x0)
    /home/david/go-git/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc208181f80, 0xc2080c2000, 0x1000, 0x1000, 0xc207ffcdda, 0x0, 0x0)
    <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080a1980)
    /home/david/go-git/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080a1980, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /home/david/go-git/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20817a790)
    /home/david/go-git/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
    /home/david/go-git/src/net/http/transport.go:660 +0xc9f

goroutine 259 [select]:
net/http.(*persistConn).roundTrip(0xc20817a790, 0xc20819e420, 0x0, 0x0, 0x0)
    /home/david/go-git/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc208076000, 0xc2081fd1e0, 0xc208181e60, 0x0, 0x0)
    /home/david/go-git/src/net/http/transport.go:235 +0x558
net/http.send(0xc2081fd1e0, 0x7ff8d880c468, 0xc208076000, 0x17, 0x0, 0x0)
    /home/david/go-git/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xa395a0, 0xc2081fd1e0, 0x17, 0x0, 0x0)
    /home/david/go-git/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xa395a0, 0xc2081fd1e0, 0x8fd318, 0x0, 0x0, 0x0)
    /home/david/go-git/src/net/http/client.go:367 +0xb25
net/http.(*Client).Get(0xa395a0, 0xc208181e00, 0x17, 0xc2081fc0d0, 0x0, 0x0)
    /home/david/go-git/src/net/http/client.go:299 +0xba
net/http.Get(0xc208181e00, 0x17, 0xc20805be58, 0x0, 0x0)
    /home/david/go-git/src/net/http/client.go:276 +0x50
net/http_test.TestLinuxSendfile(0xc2080774d0)
    /home/david/go-git/src/net/http/fs_test.go:822 +0xb14
testing.tRunner(0xc2080774d0, 0xa378a0)
    /home/david/go-git/src/testing/testing.go:447 +0xbf
created by testing.RunTests
    /home/david/go-git/src/testing/testing.go:555 +0xa8b

goroutine 263 [select]:
net/http.(*persistConn).writeLoop(0xc20817a790)
    /home/david/go-git/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
    /home/david/go-git/src/net/http/transport.go:661 +0xcbc
FAIL    net/http    120.018s

@ianlancetaylor ianlancetaylor added this to the Go1.4.2 milestone Jan 28, 2015

@ianlancetaylor ianlancetaylor changed the title Testing timeout on Go1.4.1 net/http: Testing timeout on Go1.4.1 Jan 28, 2015

@ianlancetaylor

This comment has been minimized.

Sign in to view
Copy link
Contributor

commented Jan 28, 2015

cc @bradfitz
@adg

This comment has been minimized.

Sign in to view
Copy link
Contributor

commented Jan 28, 2015

It would help to know more about your system. What OS? What hardware?
@daviddengcn

This comment has been minimized.

Sign in to view
Copy link
Author

commented Jan 28, 2015

It is a linode host. Hope this helps:

$ uname -a
Linux li69-153 3.15.3-x86_64-linode44 #1 SMP Tue Jul 1 15:51:04 EDT 2014 x86_64 GNU/Linux
$ gcc --version
gcc (Ubuntu 4.4.3-4ubuntu5.1) 4.4.3

Glad to provide more information that helps. Just tell me the linux command to run.
@adg

This comment has been minimized.

Sign in to view
Copy link
Contributor

commented Jan 28, 2015

Does it consistently fail? Is the machine really low specced? If the machine is just very slow, then tests can timeout sometimes.
@daviddengcn

This comment has been minimized.

Sign in to view
Copy link
Author

commented Jan 28, 2015

It consistently failed on my machine. I just killed all the background processes that may consume resource and ran it again. The testcase was still timeout. All other cases were finished within 2s.

@adg adg modified the milestones: Go1.4.3, Go1.4.2 Feb 17, 2015

@alexcesaro

This comment has been minimized.

Sign in to view
Copy link
Contributor

commented Feb 26, 2015

I also got this test failing on a Docker container (Ubuntu 12.04) while the test pass outside the container. It fails on tags go1.4 and go1.4.2.

http.Get timeouts here: https://github.com/golang/go/blob/release-branch.go1.4/src/net/http/fs_test.go#L822
@alexcesaro

This comment has been minimized.

Sign in to view
Copy link
Contributor

commented Feb 27, 2015

I just had a look and the test fails because strace returns an error: strace: ptrace(PTRACE_TRACEME, ...): Permission denied

It happens because of a ptrace protection: https://wiki.ubuntu.com/SecurityTeam/Roadmap/KernelHardening#ptrace_Protection

Since strace is run asynchronously, the test does not see the error and http.Get timeouts. So there is two ways to fix that issue:

  • checking the content of /proc/sys/kernel/yama/ptrace_scope
  • using child.Run() in a goroutine instead of child.Start()

I think the second fix is the best one. I can send a CL if needed.

@t-yuki t-yuki referenced this issue Mar 11, 2015

Closed

can't parse `autogenerated` function #1

@ianlancetaylor ianlancetaylor modified the milestones: Go1.5, Go1.4.3 Jul 14, 2015

@rsc rsc modified the milestones: Unplanned, Go1.5 Jul 27, 2015

@bradfitz bradfitz changed the title net/http: Testing timeout on Go1.4.1 net/http: ptrace protections result in hung sendfile test Mar 28, 2016

@bradfitz

This comment has been minimized.

Sign in to view
Copy link
Member

commented Feb 1, 2017

@alexcesaro, feel free to send a change for Go 1.9 if you're still interested. (two years later...)

@bradfitz bradfitz modified the milestones: Go1.9, Unplanned Feb 1, 2017

@bradfitz bradfitz added help wanted Testing labels Feb 1, 2017

@kennygrant

This comment has been minimized.

Sign in to view
Copy link
Contributor

commented Mar 24, 2017 •
edited

I've tested this on docker to reproduce, and prepared a fix based on the comments by alexcesaro using child.Run(). Skips on permission failure, otherwise runs the test.

=== RUN TestLinuxSendfile
--- PASS: TestLinuxSendfile (0.02s)
=== RUN TestLinuxSendfileChild
--- PASS: TestLinuxSendfileChild (0.00s)
PASS
ok net/http 0.031s

=== RUN TestLinuxSendfile
--- SKIP: TestLinuxSendfile (0.00s)
fs_test.go:1098: skipping; failed to run strace: exit status 1
=== RUN TestLinuxSendfileChild
--- PASS: TestLinuxSendfileChild (0.00s)
PASS
ok net/http 0.011s
@gopherbot

This comment has been minimized.

Sign in to view
Copy link

commented Mar 24, 2017

CL https://golang.org/cl/38633 mentions this issue.

@gopherbot gopherbot closed this in 34396ad Mar 24, 2017

@gopherbot

This comment has been minimized.

Sign in to view
Copy link

commented Oct 20, 2017

Change https://golang.org/cl/72170 mentions this issue: [release-branch.go1.8] net/http: Fix TestLinuxSendfile without strace permissions

gopherbot pushed a commit that referenced this issue Oct 25, 2017

@kennygrant @rsc
[release-branch.go1.8] net/http: Fix TestLinuxSendfile without strace… 
… permissions

If go doesn't have permission to run strace, this test hangs while
waiting for strace to run. Instead try invoking strace with
Run() first - on fail skip and report error, otherwise run
the test normally using strace.

Also fix link to open mips64 issue in same test.

Fixes #9711

Change-Id: Ibbc5fbb143ea6d0f8b6cfdca4b385ef4c8960b3d
Reviewed-on: https://go-review.googlesource.com/38633
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-on: https://go-review.googlesource.com/72170
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
ead964d

@golang golang locked and limited conversation to collaborators Oct 20, 2018

@gopherbot gopherbot added the FrozenDueToAge label Oct 20, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.