Skip to content

/ go

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net/http/pprof: occasional failures in TestDeltaProfile #38544

Open
bcmills opened this issue Apr 20, 2020 · 10 comments
Open

net/http/pprof: occasional failures in TestDeltaProfile #38544

bcmills opened this issue Apr 20, 2020 · 10 comments
Labels
NeedsInvestigation
Milestone
Go1.17

Comments

@bcmills
Copy link
Member

@bcmills bcmills commented Apr 20, 2020

TestDeltaProfile, added in CL 147598 for #23401, seems to be flaky on some of the arm builders.

(Perhaps it is relying on some property of scheduler latency that does not hold on the very slowest builders?)

2020-04-18T20:01:46-f5291cf/plan9-arm
2020-04-18T18:07:52-2a20f5c/android-arm-corellium

CC @hyangah @odeke-em @bradfitz

Marking as release-blocker for 1.15 since the test appears to be new for that release.
@bcmills bcmills added NeedsInvestigation release-blocker labels Apr 20, 2020
@bcmills bcmills added this to the Go1.15 milestone Apr 20, 2020
@bcmills
Copy link
Member Author

@bcmills bcmills commented Apr 22, 2020

Not just on ARM:

2020-04-22T16:45:05-141b11d/android-386-emu
2020-04-22T16:45:05-141b11d/darwin-amd64-nocgo
2020-04-20T19:36:50-0239a5c/openbsd-386-62
@bcmills bcmills changed the title net/http/pprof: TestDeltaProfile failures on arm builders net/http/pprof: occasional failures in TestDeltaProfile Apr 22, 2020
@hyangah
Copy link
Contributor

@hyangah hyangah commented Apr 22, 2020

any objection to disabling this test in all platforms other than in linux?
@bcmills
Copy link
Member Author

@bcmills bcmills commented Apr 22, 2020

Is there a reason to suspect that it is not flaky on Linux?

(In my experience, the unusual builders are often just more timing-sensitive than the mainline builders — tests that are flaky on more than one of those platforms also tend to be flaky on linux, just at a lower rate.)
@hyangah
Copy link
Contributor

@hyangah hyangah commented Apr 22, 2020

Or, maybe we completely delete this test?
I don't know what to make out of testing this on the platforms where even time.Sleep(1sec) results in sleeping more than 2 sec.
@gopherbot
Copy link

@gopherbot gopherbot commented Apr 22, 2020

Change https://golang.org/cl/229498 mentions this issue: net/http/pprof: make TestDeltaProfile less flaky by retrying
gopherbot pushed a commit that referenced this issue Apr 22, 2020
@hyangah
net/http/pprof: make TestDeltaProfile less flaky by retrying
3e342e8 
In some slow environment, the goroutine for mutexHog2 may not run
within 1secs. So, try with increasing seconds parameters,
and declare failure if it still fails with the longest duration
parameter (32sec).

Also, relax the test condition - previously we expected the
profile's duration is within 0.5~2sec. But obviously, in some
slow environment, that's not even guaranteed. Just check we get
non-zero duration in the result.

Update #38544

Change-Id: Ia9b0d51429a2093e6c9eb92cf463ff6952ef3e10
Reviewed-on: https://go-review.googlesource.com/c/go/+/229498
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@gopherbot

This comment was marked as off-topic.

Sign in to view
@josharian

This comment was marked as off-topic.

Sign in to view
@hyangah
Copy link
Contributor

@hyangah hyangah commented Apr 28, 2020

I don't see any more test failures after the cl. Closing it. Thanks!
@hyangah hyangah closed this Apr 28, 2020
@bcmills
Copy link
Member Author

@bcmills bcmills commented Jul 27, 2020

Still appears to be flaky, but removing the release-blocker label because the flake rate seems at least reduced.

2020-07-13T17:48:55-e88ea87/openbsd-arm-jsing
2020-06-23T18:38:32-9f33108/android-386-emu
2020-06-10T20:58:11-ec177e4/openbsd-arm-jsing
2020-05-29T20:55:14-c2d1df6/android-arm64-corellium
2020-05-28T14:43:19-6bf2eea/openbsd-arm-jsing
@bcmills bcmills removed the release-blocker label Jul 27, 2020
@bcmills bcmills reopened this Jul 27, 2020
@andybons andybons modified the milestones: Go1.15, Go1.16 Aug 11, 2020
@odeke-em
Copy link
Member

@odeke-em odeke-em commented Feb 6, 2021

Hasn’t been flaky for the past 6 months, so punting to Go1.17.
@odeke-em odeke-em modified the milestones: Go1.16, Go1.17 Feb 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation
Projects
None yet
Milestone
Go1.17
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
@josharian @andybons @odeke-em @hyangah @bcmills @gopherbot