Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: apparent deadlock in TestNoShrinkStackWhileParking on netbsd-arm64-bsiegert #49382

Open
bcmills opened this issue Nov 5, 2021 · 3 comments
Labels
arch-arm64 NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-NetBSD
Milestone

Comments

@bcmills
Copy link
Member

bcmills commented Nov 5, 2021

greplogs --dashboard -md -l -e runtime_test\\.TestNoShrinkStackWhileParking\\.func4

2021-11-05T07:00:05-6fefb7f/netbsd-arm64-bsiegert
2021-11-05T00:52:09-bd580a0/netbsd-arm64-bsiegert
2021-11-04T23:56:29-0e5f287/netbsd-arm64-bsiegert
2021-11-04T18:22:03-b2149ac/netbsd-arm64-bsiegert
2021-11-04T02:57:37-9e4e23c/netbsd-arm64-bsiegert
2021-11-03T22:51:09-1f368d5/netbsd-arm64-bsiegert
2021-11-02T17:31:50-4e7dd9f/netbsd-arm64-bsiegert
2021-11-02T13:43:24-4d7bf41/netbsd-arm64-bsiegert
2021-11-02T06:25:39-088bb4b/netbsd-arm64-bsiegert
2021-11-02T06:21:44-3ee426a/netbsd-arm64-bsiegert
2021-11-02T06:20:21-c1ea339/netbsd-arm64-bsiegert
2021-11-02T00:12:17-81fea0b/netbsd-arm64-bsiegert
2021-10-31T18:13:09-fd09e88/netbsd-arm64-bsiegert
2021-10-31T08:29:02-8e3d5f0/netbsd-arm64-bsiegert
2021-10-30T16:46:47-d1dceaf/netbsd-arm64-bsiegert
2021-10-29T23:15:00-8dfb447/netbsd-arm64-bsiegert
2021-10-29T22:29:31-c812b97/netbsd-arm64-bsiegert
2021-10-29T17:12:47-3aecb3a/netbsd-arm64-bsiegert
2021-10-28T22:38:00-af05d8b/netbsd-arm64-bsiegert
2021-10-28T15:35:25-278b9b3/netbsd-arm64-bsiegert
2021-10-27T18:24:50-259735f/netbsd-arm64-bsiegert
2021-10-27T16:59:43-c0ac39c/netbsd-arm64-bsiegert
2021-10-27T06:23:35-5786a54/netbsd-arm64-bsiegert
2021-10-26T23:06:10-e5c5125/netbsd-arm64-bsiegert
2021-10-26T19:04:53-1dc77a3/netbsd-arm64-bsiegert
2021-10-26T14:05:47-a2b8c18/netbsd-arm64-bsiegert
2021-10-25T12:07:15-fcd2d9c/netbsd-arm64-bsiegert
2021-10-23T20:23:35-8dbf3e9/netbsd-arm64-bsiegert
2021-10-23T16:18:19-16318df/netbsd-arm64-bsiegert
2021-10-23T16:17:58-3cd28ba/netbsd-arm64-bsiegert
2021-10-21T23:48:53-3beab0c/netbsd-arm64-bsiegert
2021-10-21T20:12:09-5a3d0f5/netbsd-arm64-bsiegert
2021-10-21T17:42:54-8151b56/netbsd-arm64-bsiegert
2021-10-21T17:31:42-392bb06/netbsd-arm64-bsiegert
2021-10-20T20:39:36-016d5ee/netbsd-arm64-bsiegert
2021-10-20T20:38:59-75b73d6/netbsd-arm64-bsiegert
2021-10-20T19:30:13-3ff39c5/netbsd-arm64-bsiegert
2021-10-20T00:05:22-4cc6a91/netbsd-arm64-bsiegert
2021-10-19T22:47:48-2be5b84/netbsd-arm64-bsiegert
2021-10-18T22:56:07-eba0e86/netbsd-arm64-bsiegert
2021-10-18T22:55:05-7397178/netbsd-arm64-bsiegert
2021-10-17T04:27:13-cf51fb5/netbsd-arm64-bsiegert
2021-10-17T02:23:16-fa7d11a/netbsd-arm64-bsiegert
2021-10-16T16:27:40-640a49b/netbsd-arm64-bsiegert
2021-10-15T19:03:33-cfe6763/netbsd-arm64-bsiegert
2021-10-15T18:07:49-8331f25/netbsd-arm64-bsiegert
2021-10-15T16:57:36-a80e53e/netbsd-arm64-bsiegert
2021-10-15T03:49:17-8dab959/netbsd-arm64-bsiegert
2021-10-15T02:02:50-3da0ff8/netbsd-arm64-bsiegert
2021-10-14T23:32:09-9e8ed86/netbsd-arm64-bsiegert
2021-10-14T15:16:55-ad99d88/netbsd-arm64-bsiegert
2021-10-13T20:31:25-b90d258/netbsd-arm64-bsiegert
2021-10-13T18:15:48-40f82f8/netbsd-arm64-bsiegert
2021-10-13T17:16:12-4a3daee/netbsd-arm64-bsiegert
2021-10-13T16:41:20-aded167/netbsd-arm64-bsiegert
2021-10-13T16:36:59-69041c7/netbsd-arm64-bsiegert
2021-10-13T15:12:46-f687831/netbsd-arm64-bsiegert
2021-10-13T04:00:15-e8f99da/netbsd-arm64-bsiegert
2021-10-13T00:11:47-4fb2e1c/netbsd-arm64-bsiegert
2021-10-12T21:30:45-3283d1a/netbsd-arm64-bsiegert
2021-10-12T20:09:58-4186db6/netbsd-arm64-bsiegert
2021-10-12T18:52:28-26b6833/netbsd-arm64-bsiegert
2021-10-12T18:00:16-6e0adde/netbsd-arm64-bsiegert
2021-10-12T04:35:19-6372e7e/netbsd-arm64-bsiegert
2021-10-11T22:17:47-b41030e/netbsd-arm64-bsiegert
2021-10-11T19:20:12-65ffee6/netbsd-arm64-bsiegert
2021-10-08T19:41:16-7fcf9a1/netbsd-arm64-bsiegert
2021-10-08T19:30:58-f430bda/netbsd-arm64-bsiegert
2021-10-08T18:07:25-d480b5c/netbsd-arm64-bsiegert
2021-10-08T00:24:52-78d749f/netbsd-arm64-bsiegert
2021-10-07T18:02:14-019ad98/netbsd-arm64-bsiegert
2021-10-07T02:30:47-39bbf08/netbsd-arm64-bsiegert
2021-10-06T23:19:38-f375844/netbsd-arm64-bsiegert
2021-10-06T21:17:08-2e107b4/netbsd-arm64-bsiegert
2021-10-06T16:11:20-aecf4b1/netbsd-arm64-bsiegert
2021-10-05T23:36:43-e82ed0c/netbsd-arm64-bsiegert
2021-10-05T23:25:24-77bd0da/netbsd-arm64-bsiegert
2021-10-05T23:25:06-e31c9ab/netbsd-arm64-bsiegert
2021-10-05T22:42:11-96fface/netbsd-arm64-bsiegert
2021-10-05T21:22:16-8444a54/netbsd-arm64-bsiegert
2021-10-05T20:53:02-695a59b/netbsd-arm64-bsiegert
2021-10-04T23:29:20-e5f6d8d/netbsd-arm64-bsiegert
2021-10-04T17:17:11-cc5e3de/netbsd-arm64-bsiegert
2021-10-04T15:50:43-9432320/netbsd-arm64-bsiegert
2021-10-02T17:29:46-3bbc823/netbsd-arm64-bsiegert
2021-10-02T13:52:59-07a7c6f/netbsd-arm64-bsiegert
2021-10-02T00:31:26-64da5e0/netbsd-arm64-bsiegert
2021-10-01T17:27:24-952df98/netbsd-arm64-bsiegert
2021-10-01T15:59:38-e5ad363/netbsd-arm64-bsiegert
2021-09-30T19:57:19-8b2f583/netbsd-arm64-bsiegert
2021-09-30T16:43:32-9112d29/netbsd-arm64-bsiegert
2021-09-30T16:34:25-d4aed7e/netbsd-arm64-bsiegert
2021-09-29T22:14:44-2d6d558/netbsd-arm64-bsiegert
2021-09-29T20:45:11-40fa8c2/netbsd-arm64-bsiegert
2021-09-29T18:38:00-82ac9ab/netbsd-arm64-bsiegert
2021-09-29T16:45:42-2fad7db/netbsd-arm64-bsiegert
2021-09-28T20:40:48-daec057/netbsd-arm64-bsiegert
2021-09-28T17:50:53-8cf0a08/netbsd-arm64-bsiegert
2021-09-27T22:22:35-cd4d592/netbsd-arm64-bsiegert
2021-09-27T22:22:04-52b23a5/netbsd-arm64-bsiegert
2021-09-27T21:31:14-04f7521/netbsd-arm64-bsiegert
2021-09-27T20:58:24-6c163e5/netbsd-arm64-bsiegert
2021-09-27T20:42:34-301f6c8/netbsd-arm64-bsiegert
2021-09-27T16:54:36-f9a53b6/netbsd-arm64-bsiegert
2021-09-27T11:52:07-6e5dd0b/netbsd-arm64-bsiegert
2021-09-27T05:10:56-54079df/netbsd-arm64-bsiegert
2021-09-25T01:24:46-8854368/netbsd-arm64-bsiegert
2021-09-24T19:40:03-cfd0868/netbsd-arm64-bsiegert
2021-09-24T17:52:54-584afc2/netbsd-arm64-bsiegert
2021-09-24T01:18:01-0f1159b/netbsd-arm64-bsiegert
2021-09-24T00:30:36-c90ead9/netbsd-arm64-bsiegert
2021-09-23T19:41:45-ddb5a42/netbsd-arm64-bsiegert
2021-09-23T02:52:10-d0dd26a/netbsd-arm64-bsiegert
2021-09-22T23:17:55-ee69b09/netbsd-arm64-bsiegert
2021-09-22T22:50:41-333f3de/netbsd-arm64-bsiegert
2021-09-22T00:15:27-04572fa/netbsd-arm64-bsiegert
2021-09-21T23:11:47-30faf96/netbsd-arm64-bsiegert
2021-09-17T22:41:29-50e4508/netbsd-arm64-bsiegert
2021-09-17T19:50:04-cea7a71/netbsd-arm64-bsiegert
2021-09-16T23:56:28-265b59a/netbsd-arm64-bsiegert
2021-09-16T23:13:58-e09dcc2/netbsd-arm64-bsiegert
2021-09-15T22:53:42-cfa233d/netbsd-arm64-bsiegert
2021-09-15T22:07:58-0edc6c4/netbsd-arm64-bsiegert
2021-09-15T20:29:43-03df68d/netbsd-arm64-bsiegert
2021-09-15T16:32:27-72bb818/netbsd-arm64-bsiegert
2021-09-15T14:17:48-5b48fca/netbsd-arm64-bsiegert
2021-09-15T09:43:00-e4dfd78/netbsd-arm64-bsiegert
2021-09-15T03:29:46-4847c47/netbsd-arm64-bsiegert
2021-09-14T14:09:54-8699425/netbsd-arm64-bsiegert
2021-09-14T02:52:48-2953cd0/netbsd-arm64-bsiegert
2021-09-14T00:49:39-4a4221e/netbsd-arm64-bsiegert
2021-09-12T01:06:53-0d8a4bf/netbsd-arm64-bsiegert
2021-09-11T05:05:56-23832ba/netbsd-arm64-bsiegert
2021-09-11T04:44:12-a50225a/netbsd-arm64-bsiegert
2021-09-09T22:20:16-1a708bc/netbsd-arm64-bsiegert
2021-09-09T22:17:54-73483df/netbsd-arm64-bsiegert
2021-09-09T04:37:47-376a079/netbsd-arm64-bsiegert
2021-09-05T23:10:46-7619a45/netbsd-arm64-bsiegert
2021-09-04T19:33:54-28dae3d/netbsd-arm64-bsiegert
2021-09-04T19:33:36-37e9c1d/netbsd-arm64-bsiegert
2021-09-04T15:01:53-5c224ec/netbsd-arm64-bsiegert
2021-09-04T10:59:00-ba66d62/netbsd-arm64-bsiegert
2021-09-04T10:42:34-9cb5716/netbsd-arm64-bsiegert
2021-09-03T15:32:47-52aef05/netbsd-arm64-bsiegert
2021-09-02T19:16:19-9633195/netbsd-arm64-bsiegert
2021-09-02T17:30:32-7609b50/netbsd-arm64-bsiegert
2021-09-02T15:05:39-d8b1567/netbsd-arm64-bsiegert
2021-09-02T12:49:05-08e2519/netbsd-arm64-bsiegert
2021-09-02T09:24:44-d13d62c/netbsd-arm64-bsiegert
2021-09-01T07:10:24-717f337/netbsd-arm64-bsiegert
2021-08-31T08:41:16-f118d14/netbsd-arm64-bsiegert
2021-08-30T22:07:53-3342aa5/netbsd-arm64-bsiegert
2021-08-30T21:51:52-86fa510/netbsd-arm64-bsiegert
2021-08-30T16:00:10-7b38dd8/netbsd-arm64-bsiegert
2021-08-30T02:40:35-21d0b30/netbsd-arm64-bsiegert
2021-08-30T01:55:58-a29d9aa/netbsd-arm64-bsiegert
2021-08-29T14:10:21-f29abcc/netbsd-arm64-bsiegert
2021-08-28T17:05:43-6df3aac/netbsd-arm64-bsiegert
2021-08-28T01:26:53-ef4cb2f/netbsd-arm64-bsiegert
2021-08-27T16:41:29-6a35e07/netbsd-arm64-bsiegert
2021-08-27T05:13:44-2c60a99/netbsd-arm64-bsiegert
2021-08-27T01:42:38-d7e2e2e/netbsd-arm64-bsiegert
2021-08-26T19:34:58-eb6a07f/netbsd-arm64-bsiegert
2021-08-26T19:34:52-3836983/netbsd-arm64-bsiegert
2021-08-26T19:07:27-1f8d456/netbsd-arm64-bsiegert
2021-08-25T19:45:20-6cf1d5d/netbsd-arm64-bsiegert
2021-08-25T18:27:41-d2f002c/netbsd-arm64-bsiegert
2021-08-24T16:36:55-b1cdf86/netbsd-arm64-bsiegert
2021-08-23T19:49:23-0a7f00a/netbsd-arm64-bsiegert
2021-08-23T19:46:36-6b9e3f8/netbsd-arm64-bsiegert
2021-08-23T13:09:11-7a6d64f/netbsd-arm64-bsiegert
2021-08-23T03:39:14-457418b/netbsd-arm64-bsiegert
2021-08-19T16:10:32-91e2e3b/netbsd-arm64-bsiegert
2021-08-18T21:19:22-c2bd9ee/netbsd-arm64-bsiegert
2021-08-18T15:11:52-0c83e01/netbsd-arm64-bsiegert
2021-08-18T08:26:44-8b471db/netbsd-arm64-bsiegert
2021-08-18T02:08:09-946e254/netbsd-arm64-bsiegert
2021-08-17T21:34:32-a2a9a7b/netbsd-arm64-bsiegert
2021-08-17T16:22:15-cf12b0d/netbsd-arm64-bsiegert

@bsiegert, @coypoop: could you investigate? This is the only builder exhibiting this failure, and it occurs quite frequently.

@bcmills
Copy link
Member Author

bcmills commented Nov 5, 2021

(Note that this failure mode appears to predate #49209.)

@bcmills bcmills added this to the Backlog milestone Nov 5, 2021
@gopherbot
Copy link

gopherbot commented Nov 5, 2021

Change https://golang.org/cl/361615 mentions this issue: runtime: skip TestNoShrinkStackWhileParking on netbsd/arm64

@bcmills
Copy link
Member Author

bcmills commented Nov 5, 2021

This is a release-blocker (via #11811) until the test is skipped. (Then it can go in the backlog, because netbsd/arm64 is not a first-class port.)

@bcmills bcmills modified the milestones: Backlog, Go1.18 Nov 5, 2021
@bcmills bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. and removed release-blocker labels Nov 5, 2021
@bcmills bcmills modified the milestones: Go1.18, Backlog Nov 5, 2021
gopherbot pushed a commit that referenced this issue Nov 5, 2021
This test appears to deadlock frequently on the only netbsd-arm64
builder we have (netbsd-arm64-bsiegert). Skip the test to provide
more useful test coverage for other failures.

For #49382

Change-Id: I3be32f58ce1e396f7c69163e70cf58f779f57ac6
Reviewed-on: https://go-review.googlesource.com/c/go/+/361615
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Benny Siegert <bsiegert@gmail.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
matzf added a commit to matzf/net that referenced this issue Dec 1, 2021
TestUDP/Messages and TestUDP/Messages-dialed occasionally failed because
the expected messages were not received in a single RecvMsgs call, or
because the messages were received out of order.

Assuming that both messages are returned immediately from a single
RecvMsgs call was a flawed expectation. Fixed by repeatedly invoking
RecvMsgs until all expected messages have been received.

While it certainly seems unusual that packets are reordered on a
loopback device, it does appear to happen occasionally (on linux-mips).
Fixed by sizing receive buffers such that messages in any order can be
received correctly, and by allowing either order for the reassembled
message.

Combine "Messages" and "Messages-dialed" subtests with a simple
table-driven test, to avoid the repetition. The same "Message" and
"Message-dialed".
Finally, make the test failure messages slightly more useful.

Fixes golang/go#49382
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arch-arm64 NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-NetBSD
Projects
None yet
Development

No branches or pull requests

2 participants