-
Notifications
You must be signed in to change notification settings - Fork 17.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cmd/cgo/internal/testcarchive: TestManyCalls failures #67566
Comments
Found new dashboard test flakes for:
2024-05-21 22:34 gotip-linux-amd64-longtest go@4b778470 cmd/cgo/internal/testcarchive.TestManyCalls (log)
|
Found new dashboard test flakes for:
2024-05-22 09:58 gotip-linux-amd64-longtest go@cd41d717 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-05-22 14:20 gotip-linux-amd64-longtest go@1d3d7d39 cmd/cgo/internal/testcarchive.TestManyCalls (log)
|
There are more flaky failures on this since June 20, https://ci.chromium.org/ui/test/golang/cmd%2Fcgo%2Finternal%2Ftestcarchive.TestManyCalls?q=V%3Abuilder%3Dgotip-linux-amd64-longtest-test_only+V%3Ago_branch%3Dmaster+V%3Agoarch%3Damd64+V%3Agoos%3Dlinux+V%3Ahost_goarch%3Damd64+V%3Ahost_goos%3Dlinux Tentatively mark as a release blocker, as the high-rate flaky failures seem to start recently. |
Found new dashboard test flakes for:
2024-06-21 19:58 gotip-linux-386-longtest go@d79c3509 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-06-21 19:58 gotip-linux-amd64-longtest go@d79c3509 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-06-21 20:30 gotip-linux-amd64-longtest go@cb3b3434 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-06-22 00:00 gotip-linux-amd64-longtest go@44f18706 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-06-24 15:28 gotip-linux-amd64-longtest go@085cf0fc cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-06-24 15:33 gotip-linux-amd64-longtest go@0def9d5c cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-06-24 16:39 gotip-linux-386-longtest go@b98803e8 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-06-24 16:39 gotip-linux-amd64-longtest go@b98803e8 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-06-24 17:52 go1.22-linux-amd64-longtest release-branch.go1.22@3560cf0a cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-06-24 18:55 gotip-linux-amd64-longtest go@f214a760 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-06-24 21:07 gotip-linux-amd64-longtest go@68315bc8 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-06-26 16:10 gotip-linux-amd64-longtest go@a2e90be9 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-06-27 17:41 gotip-linux-amd64-longtest go@ea537cca cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-06-28 15:25 gotip-linux-amd64-longtest go@7f90b960 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-06-28 19:59 gotip-linux-amd64-longtest go@82c371a3 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-01 14:58 gotip-linux-amd64-longtest go@d3c93f2f cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-02 18:51 go1.22-linux-amd64-longtest release-branch.go1.22@8e1fdea8 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-03 00:14 gotip-linux-amd64-longtest go@148755a2 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-03 13:03 gotip-linux-amd64-longtest go@71f9dbb1 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-07 16:40 gotip-linux-amd64-longtest go@d0146bd8 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-07 19:43 gotip-linux-amd64-longtest go@b43d6c57 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-08 00:05 gotip-linux-amd64-longtest go@6d89b38e cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-10 02:44 gotip-linux-amd64-longtest go@70e453b4 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-10 03:19 gotip-linux-386-longtest go@b3040679 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-10 19:27 go1.22-linux-amd64-longtest release-branch.go1.22@49906f95 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-10 19:29 go1.22-linux-amd64-longtest release-branch.go1.22@45f9ded1 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-11 15:01 gotip-linux-amd64-longtest go@08a6e080 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-11 15:01 gotip-linux-amd64-longtest go@611f18c4 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-11 17:23 gotip-linux-amd64-longtest go@a71bb570 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-12 22:42 gotip-linux-amd64-longtest go@071b8d51 cmd/cgo/internal/testcarchive.TestManyCalls (log)
|
Found new dashboard test flakes for:
2024-07-09 16:14 go1.23-linux-386-longtest release-branch.go1.23@e89e880e cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-09 16:14 go1.23-linux-amd64-longtest release-branch.go1.23@e89e880e cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-12 22:42 go1.23-linux-amd64-longtest release-branch.go1.23@071b8d51 cmd/cgo/internal/testcarchive.TestManyCalls (log)
|
I pulled the test timing data from LUCI. For the first elevated period, as afbbc28 is a revert of d881ed6, it is likely that d881ed6 is the cause and the revert fixes it. For the second elevation, d79c350 is unrelated, so are a few commits nearby. So it is probably something else. Maybe builder change? Then I pulled the test timing data for the 1.22 release branch, which doesn't have any of those commits. Lastly, the test has a hard-coded 1 minute timeout. I don't think it makes much sense to say 55 seconds is pass, 65 seconds is fail. So the 1 minute timeout may be a bit arbitrary. We probably want to replace it with t.Deadline. |
Found new dashboard test flakes for:
2024-07-16 18:02 gotip-linux-amd64-longtest go@5c7f5413 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-16 18:17 gotip-linux-amd64-longtest go@b4a92f56 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-16 18:17 gotip-linux-amd64-longtest go@97ccc224 cmd/cgo/internal/testcarchive.TestManyCalls (log)
|
Change https://go.dev/cl/599056 mentions this issue: |
The 1-minute test timeout seems a bit arbitrary. Use internal/testenv.Command, which applies t.Deadline to subcommand. For #67566. Change-Id: If84c96e353bdfaf02c9b123758198e031305ae32 Reviewed-on: https://go-review.googlesource.com/c/go/+/599056 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
CL https://go.dev/cl/599056 replaced the 1-minute timeout with the test deadline. Hopefully this will stop the failures on the builder. At this point I think the test slowdown is more likely due to builder infrastructure change, not the code change in the main repo (which we're going to release). Also, the test was written to catch deadlocks, not speed. Finishing in more than a minute is still deadlock. So at this point I think it is not a release blocker. It would be great we can understand the slowdown, though. So keep the issue open. |
Found new dashboard test flakes for:
2024-07-17 15:08 gotip-linux-amd64-longtest go@fc51e502 cmd/cgo/internal/testcarchive.TestManyCalls (log)
|
Found new dashboard test flakes for:
2024-07-15 21:12 go1.23-linux-amd64-longtest release-branch.go1.23@959b3fd4 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-16 15:33 go1.23-linux-386-longtest release-branch.go1.23@30b6fd60 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-16 15:33 go1.23-linux-amd64-longtest release-branch.go1.23@30b6fd60 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-16 18:03 go1.23-linux-amd64-longtest release-branch.go1.23@d5479e19 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-16 18:17 go1.23-linux-amd64-longtest release-branch.go1.23@b4a92f56 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-16 18:17 go1.23-linux-amd64-longtest release-branch.go1.23@0dae393a cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-16 18:17 go1.23-linux-amd64-longtest release-branch.go1.23@66e940b6 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-16 18:17 go1.23-linux-amd64-longtest release-branch.go1.23@97ccc224 cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-07-17 15:08 go1.23-linux-amd64-longtest release-branch.go1.23@fc51e502 cmd/cgo/internal/testcarchive.TestManyCalls (log)
|
Found new dashboard test flakes for:
2024-07-17 22:41 go1.22-linux-amd64-longtest release-branch.go1.22@4c50f916 cmd/cgo/internal/testcarchive.TestManyCalls (log)
|
All failures are on the release branches before CL 599056. |
I'll double-check that there weren't any builder configuration changes around the dates where the test time went up really high. |
Found new dashboard test flakes for:
2024-07-31 16:13 go1.22-linux-amd64-longtest release-branch.go1.22@8c8adffd cmd/cgo/internal/testcarchive.TestManyCalls (log)
|
Found new dashboard test flakes for:
2024-08-14 17:44 go1.22-linux-amd64-longtest release-branch.go1.22@0a525a3e cmd/cgo/internal/testcarchive.TestManyCalls (log)
2024-08-21 18:15 go1.22-linux-amd64-longtest release-branch.go1.22@676d6100 cmd/cgo/internal/testcarchive.TestManyCalls (log)
|
Found new dashboard test flakes for:
2024-09-05 22:14 go1.22-linux-amd64-longtest release-branch.go1.22@71655f14 cmd/cgo/internal/testcarchive.TestManyCalls (log)
|
@gopherbot Please backport to 1.22 and 1.23. This is a test-only fix for a flaky test that is regularly flaky on the release branches. |
Backport issue(s) opened: #70124 (for 1.22), #70125 (for 1.23). Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases. |
Change https://go.dev/cl/623595 mentions this issue: |
…e timeout The 1-minute test timeout seems a bit arbitrary. Use internal/testenv.Command, which applies t.Deadline to subcommand. For #67566. Fixes #70124. Change-Id: If84c96e353bdfaf02c9b123758198e031305ae32 Reviewed-on: https://go-review.googlesource.com/c/go/+/599056 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> (cherry picked from commit 8c88f0c) Reviewed-on: https://go-review.googlesource.com/c/go/+/623595 Reviewed-by: Cherry Mui <cherryyz@google.com>
Issue created automatically to collect these failures.
Example (log):
— watchflakes
The text was updated successfully, but these errors were encountered: