Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: OpenBSD wirep related failures #49259

Open
mknyszek opened this issue Nov 1, 2021 · 12 comments
Open

runtime: OpenBSD wirep related failures #49259

mknyszek opened this issue Nov 1, 2021 · 12 comments

Comments

@mknyszek
Copy link
Contributor

@mknyszek mknyszek commented Nov 1, 2021

On trybots and in CI I've noticed a recent uptick in failures on OpenBSD, looking like

2021-11-01T02:47:30-fde4cc2/openbsd-amd64-68
2021-10-29T22:29:31-c812b97/openbsd-386-68
2021-10-28T03:44:12-2ff1074/openbsd-amd64-68

At first I thought this was one of my CLs, but the oldest one seems to indicate it's something else.

There are two other older ones that may be related:

2021-10-26T05:05:24-23fdd7f/openbsd-386-68
2021-10-26T01:22:47-adfb85b/openbsd-386-68

@mknyszek mknyszek added this to the Go1.18 milestone Nov 1, 2021
@mknyszek
Copy link
Contributor Author

@mknyszek mknyszek commented Nov 1, 2021

Loading

@bcmills
Copy link
Member

@bcmills bcmills commented Nov 2, 2021

The wirep failure mode may be new, but the OpenBSD runtime crashes are very much not. I suspect that this is a new manifestation of some existing source of memory corruption.

greplogs --dashboard -md -l -e '(?ms)\Aopenbsd-386.*fatal error:' --since=2021-06-01

2021-11-01T13:08:16-732db40/openbsd-386-68
2021-10-29T22:29:31-c812b97/openbsd-386-68
2021-10-26T05:05:24-23fdd7f/openbsd-386-68
2021-10-26T01:22:47-adfb85b/openbsd-386-68
2021-10-21T17:03:18-4100dac-6ec9a1d/openbsd-386-64
2021-10-21T17:03:18-4100dac-2ac3bdf/openbsd-386-64
2021-10-21T15:45:05-9b675d0-6ec9a1d/openbsd-386-64
2021-10-21T13:05:07-9b675d0-8b9c0d1/openbsd-386-64
2021-10-21T02:35:47-9b675d0-fca46d0/openbsd-386-64
2021-10-20T20:58:15-9b675d0-73652af/openbsd-386-64
2021-10-20T20:41:51-9b675d0-2ac3bdf/openbsd-386-64
2021-10-20T18:48:51-cbf1d01-2ac3bdf/openbsd-386-64
2021-10-20T07:21:22-4ea6123-d9421ce/openbsd-386-64
2021-10-20T05:28:45-4ea6123-b091189/openbsd-386-64
2021-10-20T00:05:22-4ea6123-4cc6a91/openbsd-386-64
2021-10-19T23:27:41-4ea6123-c580180/openbsd-386-64
2021-10-19T23:13:35-81f084e-2be5b84/openbsd-386-64
2021-10-19T23:13:35-81f084e-2ac3bdf/openbsd-386-64
2021-10-19T22:56:52-3a9f390-c580180/openbsd-386-64
2021-10-19T22:56:52-3a9f390-2ac3bdf/openbsd-386-64
2021-10-19T21:17:55-9f721e8-c580180/openbsd-386-64
2021-10-19T21:17:55-9f721e8-2ac3bdf/openbsd-386-64
2021-10-19T21:05:11-1feb683-1b24c9e/openbsd-386-64
2021-10-19T20:35:15-1feb683-c580180/openbsd-386-64
2021-10-19T20:35:15-1feb683-07e5527/openbsd-386-64
2021-10-19T18:22:59-98f6e03-6c3cd5d/openbsd-386-64
2021-10-19T17:12:53-98f6e03-404f84d/openbsd-386-64
2021-10-19T17:12:53-404f84d/openbsd-386-68
2021-10-19T17:01:56-98f6e03-99fad12/openbsd-386-64
2021-10-19T16:19:27-98f6e03-f92a358/openbsd-386-64
2021-10-19T15:48:56-98f6e03-067d796/openbsd-386-64
2021-10-19T15:03:39-98f6e03-6294207/openbsd-386-64
2021-10-19T09:49:38-98f6e03-b0351bf/openbsd-386-64
2021-10-19T08:44:22-98f6e03-1e49210/openbsd-386-64
2021-10-18T22:56:07-98f6e03-eba0e86/openbsd-386-64
2021-10-18T21:21:54-98f6e03-6c0daa7/openbsd-386-64
2021-10-18T20:39:22-98f6e03-267abbe/openbsd-386-64
2021-10-18T19:46:27-98f6e03-394a1ad/openbsd-386-64
2021-10-17T04:27:13-98f6e03-cf51fb5/openbsd-386-64
2021-10-16T16:27:40-98f6e03-640a49b/openbsd-386-64
2021-10-15T19:54:44-98f6e03-22951fb/openbsd-386-64
2021-10-15T19:03:33-98f6e03-cfe6763/openbsd-386-64
2021-10-15T16:00:01-98f6e03-fad4a16/openbsd-386-64
2021-10-15T05:20:52-288d0b3-81484cf/openbsd-386-64
2021-10-15T03:49:17-288d0b3-8dab959/openbsd-386-64
2021-10-15T02:45:22-288d0b3-3da0ff8/openbsd-386-64
2021-10-15T02:02:50-ce04ca3-3da0ff8/openbsd-386-64
2021-10-15T01:35:56-ce04ca3-0c45ed0/openbsd-386-64
2021-10-15T00:16:20-ce04ca3-9e8ed86/openbsd-386-64
2021-10-14T22:21:32-3a269dc-c580180/openbsd-386-64
2021-10-14T22:21:32-3a269dc-2ac3bdf/openbsd-386-64
2021-10-14T17:38:39-e69ba9d-011fd00/openbsd-386-64
2021-10-14T15:17:03-e69ba9d-fbdf830/openbsd-386-64
2021-10-14T14:04:14-e69ba9d-2ac3bdf/openbsd-386-64
2021-10-14T14:04:14-e69ba9d-24e798e/openbsd-386-64
2021-10-14T13:09:28-88e80df-24e798e/openbsd-386-64
2021-10-14T12:38:31-88e80df-c580180/openbsd-386-64
2021-10-14T12:38:31-88e80df-2ac3bdf/openbsd-386-64
2021-10-14T07:19:29-acac939-2feb2cc/openbsd-386-64
2021-10-14T04:18:44-acac939-9e4dc6f/openbsd-386-64
2021-10-14T04:18:44-09e6c7a-9e4dc6f/openbsd-386-64
2021-10-14T01:51:22-acac939-276fb27/openbsd-386-64
2021-10-14T00:44:41-acac939-2ac3bdf/openbsd-386-64
2021-10-13T22:21:08-ac2ed98-b90d258/openbsd-386-64
2021-10-13T22:21:08-ac2ed98-2ac3bdf/openbsd-386-64
2021-10-13T20:31:25-378b9e1-b90d258/openbsd-386-64
2021-10-13T17:16:12-378b9e1-4a3daee/openbsd-386-64
2021-10-13T16:41:20-378b9e1-aded167/openbsd-386-64
2021-10-13T15:56:29-378b9e1-5326094/openbsd-386-64
2021-10-13T15:12:46-378b9e1-f687831/openbsd-386-64
2021-10-13T04:56:02-378b9e1-e8f99da/openbsd-386-64
2021-10-13T04:56:02-378b9e1-c580180/openbsd-386-64
2021-10-13T04:56:02-378b9e1-2ac3bdf/openbsd-386-64
2021-10-13T03:12:54-24389d4-c580180/openbsd-386-64
2021-10-13T00:11:47-074820e-4fb2e1c/openbsd-386-64
2021-10-12T22:26:55-074820e-78d01be/openbsd-386-64
2021-10-12T22:12:04-94178a2-732f6fa/openbsd-386-64
2021-10-12T22:12:04-94178a2-2ac3bdf/openbsd-386-64
2021-10-12T21:30:45-0b930fb-3283d1a/openbsd-386-64
2021-10-12T20:20:41-0b930fb-d032b2b/openbsd-386-64
2021-10-12T20:09:58-0b930fb-4186db6/openbsd-386-64
2021-10-12T19:23:00-0b930fb-2ac3bdf/openbsd-386-64
2021-10-12T19:23:00-0b930fb-26b6833/openbsd-386-64
2021-10-12T18:52:28-c83725b-26b6833/openbsd-386-64
2021-10-12T18:00:16-c83725b-6e0adde/openbsd-386-64
2021-10-12T06:55:50-c83725b-d887d3b/openbsd-386-64
2021-10-11T22:57:32-c83725b-d90f0b9/openbsd-386-64
2021-10-11T22:34:49-18fa840-d90f0b9/openbsd-386-64
2021-10-11T20:46:14-18fa840-7023535/openbsd-386-64
2021-10-11T16:48:22-18fa840-aa1c6f5/openbsd-386-64
2021-10-11T16:05:38-18fa840-bd41f2d/openbsd-386-64
2021-10-11T15:23:58-18fa840-2ac3bdf/openbsd-386-64
2021-10-10T13:37:52-1af23bd-b99abf3/openbsd-386-64
2021-10-09T01:04:29-ee04797-e1c294a/openbsd-386-64
2021-10-08T22:21:41-ee04797-2ac3bdf/openbsd-386-64
2021-10-08T21:38:44-c5188f2-74abcab/openbsd-386-64
2021-10-08T19:41:16-c5188f2-7fcf9a1/openbsd-386-64
2021-10-08T18:07:25-c5188f2-d480b5c/openbsd-386-64
2021-10-08T17:58:41-c5188f2-16a3cef/openbsd-386-64
2021-10-08T17:32:31-c5188f2-d7ba1d2/openbsd-386-64
2021-10-08T16:26:20-c5188f2-99c1b24/openbsd-386-64
2021-10-08T15:27:20-c5188f2-b3a4b2d/openbsd-386-64
2021-10-08T14:08:12-c5188f2-5b9206f/openbsd-386-64
2021-10-08T02:09:58-c5188f2-e74db46/openbsd-386-64
2021-10-08T01:18:01-c5188f2-8c4ea31/openbsd-386-64
2021-10-08T00:24:52-c5188f2-78d749f/openbsd-386-64
2021-10-08T00:18:29-c5188f2-7cef831/openbsd-386-64
2021-10-07T20:02:29-d477ef3-0f52292/openbsd-386-64
2021-10-07T19:49:45-d477ef3-c580180/openbsd-386-64
2021-10-07T18:58:33-d477ef3-7286502/openbsd-386-64
2021-10-07T18:45:53-d477ef3-77f2750/openbsd-386-64
2021-10-07T16:38:24-d477ef3-be571a3/openbsd-386-64
2021-10-07T14:54:47-d477ef3-4925e07/openbsd-386-64
2021-10-07T14:37:45-d477ef3-ebeab63/openbsd-386-64
2021-10-07T14:01:52-d477ef3-ecb2f23/openbsd-386-64
2021-10-07T02:30:47-d477ef3-39bbf08/openbsd-386-64
2021-10-07T00:39:17-d477ef3-812a33d/openbsd-386-64
2021-10-06T23:26:57-d477ef3-c8dd89e/openbsd-386-64
2021-10-06T23:01:01-d477ef3-4ffa2f1/openbsd-386-64
2021-10-06T22:42:28-d477ef3-4002616/openbsd-386-64
2021-10-06T22:28:59-d477ef3-b18ba59/openbsd-386-64
2021-10-06T21:06:48-d477ef3-197b328/openbsd-386-64
2021-10-06T20:30:12-d477ef3-4a37a1d/openbsd-386-64
2021-10-06T20:21:49-d477ef3-f580b75/openbsd-386-64
2021-10-06T19:59:27-d477ef3-17c513e/openbsd-386-64
2021-10-06T19:08:15-d477ef3-6487b15/openbsd-386-64
2021-10-06T17:38:01-d477ef3-415f0a3/openbsd-386-64
2021-10-05T22:42:11-2dc2755-96fface/openbsd-386-64
2021-10-05T20:53:02-2dc2755-695a59b/openbsd-386-64
2021-10-05T20:35:54-2dc2755-990c9c6/openbsd-386-64
2021-10-05T20:15:01-2dc2755-0b4d499/openbsd-386-64
2021-10-05T17:10:45-2dc2755-5140ad1/openbsd-386-64
2021-10-05T16:44:45-2dc2755-060cd73/openbsd-386-64
2021-10-05T15:58:05-81efdbc-f1f626d/openbsd-386-64
2021-10-05T15:47:57-81efdbc-017ffcd/openbsd-386-64
2021-10-05T07:32:41-81efdbc-d55009c/openbsd-386-64
2021-10-05T03:23:35-81efdbc-123393a/openbsd-386-64
2021-10-05T03:23:35-81efdbc-04242ac/openbsd-386-64
2021-10-05T00:29:40-36e7bf9-81b7ec1/openbsd-386-64
2021-10-04T23:29:20-36e7bf9-e5f6d8d/openbsd-386-64
2021-10-04T22:59:44-36e7bf9-78c2529/openbsd-386-64
2021-10-04T22:28:37-36e7bf9-752cc07/openbsd-386-64
2021-10-04T22:08:53-db89b5a-7ee4c16/openbsd-386-64
2021-10-04T22:00:53-db89b5a-d9952ff/openbsd-386-64
2021-10-04T21:10:16-db89b5a-f0db7ea/openbsd-386-64
2021-10-04T20:57:50-db89b5a-e8a85e9/openbsd-386-64
2021-10-04T19:52:26-db89b5a-579ff8b/openbsd-386-64
2021-10-04T19:52:26-db89b5a-04242ac/openbsd-386-64
2021-10-04T18:58:42-ccaa907-7d822af/openbsd-386-64
2021-10-04T18:15:09-ccaa907-9f8d558/openbsd-386-64
2021-10-04T17:17:11-ccaa907-cc5e3de/openbsd-386-64
2021-10-04T15:50:43-ccaa907-9432320/openbsd-386-64
2021-10-02T17:29:46-0ebff1a-3bbc823/openbsd-386-64
2021-10-02T16:05:55-0ebff1a-a7fe161/openbsd-386-64
2021-10-02T13:15:42-0ebff1a-7bf7bbc/openbsd-386-64
2021-10-02T00:31:26-0ebff1a-64da5e0/openbsd-386-64
2021-10-02T00:18:01-0ebff1a-cf241a3/openbsd-386-64
2021-10-01T23:51:16-0ebff1a-c129af9/openbsd-386-64
2021-10-01T21:26:43-0ebff1a-7d57324/openbsd-386-64
2021-10-01T21:26:43-0ebff1a-04242ac/openbsd-386-64
2021-10-01T20:36:29-942994f-0d65c27/openbsd-386-64
2021-10-01T20:36:29-942994f-04242ac/openbsd-386-64
2021-10-01T18:18:46-58ad7eb-0d65c27/openbsd-386-64
2021-10-01T16:45:09-58ad7eb-f19b2d5/openbsd-386-64
2021-10-01T16:45:09-58ad7eb-7d57324/openbsd-386-64
2021-10-01T16:45:09-58ad7eb-04242ac/openbsd-386-64
2021-10-01T16:29:05-3883e4a-f19b2d5/openbsd-386-64
2021-10-01T16:29:05-3883e4a-7d57324/openbsd-386-64
2021-10-01T16:29:05-3883e4a-04242ac/openbsd-386-64
2021-10-01T15:56:03-6f5fd9b-8d494b0/openbsd-386-64
2021-10-01T04:28:36-e89823e-6a79f35/openbsd-386-64
2021-09-30T21:33:51-1c35f2a-6a79f35/openbsd-386-64
2021-09-30T20:30:12-1c35f2a-c035d82/openbsd-386-64
2021-09-30T18:10:18-1c35f2a-205640e/openbsd-386-64
2021-09-30T16:43:32-1c35f2a-9112d29/openbsd-386-64
2021-09-30T16:14:49-1c35f2a-b043d23/openbsd-386-64
2021-09-30T15:20:44-1c35f2a-7162c4c/openbsd-386-64
2021-09-29T22:14:44-1c35f2a-2d6d558/openbsd-386-64
2021-09-29T20:45:11-1c35f2a-40fa8c2/openbsd-386-64
2021-09-29T20:06:10-1c35f2a-5930cff/openbsd-386-64
2021-09-29T19:22:48-1c35f2a-82ac9ab/openbsd-386-64
2021-09-29T19:22:48-1c35f2a-04242ac/openbsd-386-64
2021-09-29T19:20:39-7d467dc-82ac9ab/openbsd-386-64
2021-09-29T19:20:39-7d467dc-7d57324/openbsd-386-64
2021-09-29T17:50:31-d9648c9-a9d5ea6/openbsd-386-64
2021-09-29T17:29:23-d9648c9-c9af2bd/openbsd-386-64
2021-09-29T17:29:23-d9648c9-04242ac/openbsd-386-64
2021-09-29T17:15:06-d42c92b-c9af2bd/openbsd-386-64
2021-09-29T16:42:30-d42c92b-7d57324/openbsd-386-64
2021-09-29T16:42:30-d42c92b-04242ac/openbsd-386-64
2021-09-29T15:31:12-5dbd5e4-aeb4fba/openbsd-386-64
2021-09-29T15:31:12-5dbd5e4-04242ac/openbsd-386-64
2021-09-28T23:59:17-df9a164-435718e/openbsd-386-64
2021-09-28T23:30:19-df9a164-b8a6017/openbsd-386-64
2021-09-28T22:34:40-0df0ca0-b8a6017/openbsd-386-64
2021-09-28T22:17:57-7898fe6-b8a6017/openbsd-386-64
2021-09-28T22:17:57-7898fe6-7d57324/openbsd-386-64
2021-09-28T21:42:57-da36f79-b8a6017/openbsd-386-64
2021-09-28T20:53:23-64bbad1-7d57324/openbsd-386-64
2021-09-28T20:53:23-64bbad1-04242ac/openbsd-386-64
2021-09-28T20:40:48-b182fde-daec057/openbsd-386-64
2021-09-28T20:30:45-b182fde-04242ac/openbsd-386-64
2021-09-28T20:12:24-b182fde-e7a2573/openbsd-386-64
2021-09-28T20:12:24-b182fde-c5c1d06/openbsd-386-64
2021-09-28T17:50:53-ba6b94c-8cf0a08/openbsd-386-64
2021-09-28T00:35:10-ba6b94c-850a4ff/openbsd-386-64
2021-09-27T22:22:35-ba6b94c-cd4d592/openbsd-386-64
2021-09-27T21:53:31-ba6b94c-7887313/openbsd-386-64
2021-09-27T21:31:14-ba6b94c-04f7521/openbsd-386-64
2021-09-27T20:42:34-ba6b94c-301f6c8/openbsd-386-64
2021-09-24T21:18:44-ba6b94c-7d57324/openbsd-386-64
2021-09-24T19:46:09-939195f-cfd0868/openbsd-386-64
2021-09-24T19:46:09-939195f-c5c1d06/openbsd-386-64
2021-09-24T19:46:09-939195f-7d57324/openbsd-386-64
2021-09-24T18:21:14-6d1e33f-f6b5ffb/openbsd-386-64
2021-09-24T18:11:24-6d1e33f-812c99f/openbsd-386-64
2021-09-24T14:52:47-6d1e33f-217507e/openbsd-386-64
2021-09-24T04:59:50-6d1e33f-242d02d/openbsd-386-64
2021-09-24T04:32:28-6d1e33f-9ce601d/openbsd-386-64
2021-09-24T04:20:30-6d1e33f-aa878ee/openbsd-386-64
2021-09-24T01:25:24-6d1e33f-483533d/openbsd-386-64
2021-09-24T01:18:01-6d1e33f-0f1159b/openbsd-386-64
2021-09-23T21:59:30-6d1e33f-4e308d7/openbsd-386-64
2021-09-23T20:15:19-6d1e33f-93f964b/openbsd-386-64
2021-09-23T17:47:39-6d1e33f-2fc7697/openbsd-386-64
2021-09-23T17:04:30-6d1e33f-13f3c57/openbsd-386-64
2021-09-23T16:54:46-6d1e33f-5961134/openbsd-386-64
2021-09-22T22:01:15-c8db761-abc4f09/openbsd-386-64
2021-09-22T22:01:15-c8db761-1537f14/openbsd-386-64
2021-09-22T21:47:18-efaec4e-1537f14/openbsd-386-64
2021-09-22T20:53:48-efaec4e-41bb744/openbsd-386-64
2021-09-22T19:37:42-efaec4e-5b75a9b/openbsd-386-64
2021-09-22T18:44:36-efaec4e-7a03ca6/openbsd-386-64
2021-09-22T18:32:21-efaec4e-ccfc41e/openbsd-386-64
2021-09-22T18:10:24-efaec4e-635e493/openbsd-386-64
2021-09-22T15:50:34-efaec4e-abc4f09/openbsd-386-64
2021-09-22T15:47:37-ecbcf3a-552410f/openbsd-386-64
2021-09-22T15:00:53-b98090b-91c2318/openbsd-386-64
2021-09-22T14:28:57-b98090b-dc5b1d3/openbsd-386-64
2021-09-22T13:55:47-b98090b-09d3df0/openbsd-386-64
2021-09-22T03:45:00-b98090b-051df0d/openbsd-386-64
2021-09-22T03:36:42-b98090b-085c609/openbsd-386-64
2021-09-22T00:15:27-b98090b-04572fa/openbsd-386-64
2021-09-21T20:35:14-b98090b-abc4f09/openbsd-386-64
2021-09-21T20:13:48-0d12d39-2f70ce3/openbsd-386-64
2021-09-21T17:25:35-fe076c8-fa5c504/openbsd-386-64
2021-09-21T16:34:59-fe076c8-13aa0d8/openbsd-386-64
2021-09-21T15:38:11-fe076c8-9ab6af9/openbsd-386-64
2021-09-21T14:02:52-fe076c8-7d67f8d/openbsd-386-64
2021-09-21T13:18:09-fe076c8-39e08c6/openbsd-386-64
2021-09-21T03:32:29-fe076c8-c7543e5/openbsd-386-64
2021-09-21T03:32:29-fe076c8-abc4f09/openbsd-386-64
2021-09-21T03:32:29-fe076c8-552410f/openbsd-386-64
2021-09-21T02:24:06-91c880c-986f8ea/openbsd-386-64
2021-09-20T19:55:18-d2417d5-af72ddf/openbsd-386-64
2021-09-20T18:41:57-464a887-3c764ba/openbsd-386-64
2021-09-20T18:36:32-464a887-6268468/openbsd-386-64
2021-09-20T18:06:25-a0016a2-988f18d/openbsd-386-64
2021-09-20T15:19:44-a0016a2-1192135/openbsd-386-64
2021-09-20T15:14:47-d2417d5-6e81f78/openbsd-386-64
2021-09-20T15:08:51-a0016a2-abc4f09/openbsd-386-64
2021-09-19T23:50:45-7559231-315dbd1/openbsd-386-64
2021-09-19T16:34:52-7559231-abc4f09/openbsd-386-64
2021-09-19T16:34:52-7559231-771b8ea/openbsd-386-64
2021-09-17T18:33:15-2758b04-70493b3/openbsd-386-64
2021-09-17T16:17:54-aba0c5f-abc4f09/openbsd-386-64
2021-09-16T23:57:40-1a7ca93-8d2a9c3/openbsd-386-64
2021-09-16T23:13:58-1a7ca93-e09dcc2/openbsd-386-64
2021-09-16T19:38:19-1a7ca93-bcdc61d/openbsd-386-64
2021-09-16T17:44:44-1a7ca93-b1bedc0/openbsd-386-64
2021-09-15T23:28:10-1a7ca93-abc4f09/openbsd-386-64
2021-09-15T22:53:42-1a7ca93-cfa233d/openbsd-386-64
2021-09-15T18:56:22-1a7ca93-170a72e/openbsd-386-64
2021-09-15T17:32:52-76d4494-6196979/openbsd-386-64
2021-09-15T17:19:52-76d4494-21a4e67/openbsd-386-64
2021-09-15T09:43:47-02e5238-21a4e67/openbsd-386-64
2021-09-15T09:43:47-02e5238-170a72e/openbsd-386-64
2021-09-15T00:13:36-0cffec9-21a4e67/openbsd-386-64
2021-09-14T23:55:15-5492d01-bf26e43/openbsd-386-64
2021-09-14T23:07:15-9207707-137543b/openbsd-386-64
2021-09-14T22:36:57-9207707-170a72e/openbsd-386-64
2021-09-14T21:57:30-258ee27-b2c04f0/openbsd-386-64
2021-09-14T19:06:02-258ee27-21a4e67/openbsd-386-64
2021-09-14T19:06:02-258ee27-181e8cd/openbsd-386-64
2021-09-14T19:06:02-258ee27-170a72e/openbsd-386-64
2021-09-14T14:09:54-384e5da-8699425/openbsd-386-64
2021-09-14T13:53:08-384e5da-b3c6de9/openbsd-386-64
2021-09-14T02:53:17-384e5da-ee91bb8/openbsd-386-64
2021-09-14T00:49:39-384e5da-4a4221e/openbsd-386-64
2021-09-14T00:29:16-384e5da-71adc65/openbsd-386-64
2021-09-14T00:16:36-384e5da-146e8d4/openbsd-386-64
2021-09-14T00:08:19-384e5da-21a4e67/openbsd-386-64
2021-09-13T21:48:41-e7de652-42057e9/openbsd-386-64
2021-09-13T18:05:59-915f620-c8a58f2/openbsd-386-64
2021-09-12T01:06:53-915f620-0d8a4bf/openbsd-386-64
2021-09-11T05:05:56-915f620-23832ba/openbsd-386-64
2021-09-11T04:35:30-915f620-cf2fe5d/openbsd-386-64
2021-09-10T10:45:59-295a39b-c69f5c0/openbsd-386-64
2021-09-10T02:44:36-295a39b-2091bd3/openbsd-386-64
2021-09-09T23:18:18-295a39b-b32209d/openbsd-386-64
2021-09-09T22:20:16-295a39b-1a708bc/openbsd-386-64
2021-09-09T21:49:46-295a39b-c981874/openbsd-386-64
2021-09-09T21:49:46-295a39b-21a4e67/openbsd-386-64
2021-09-09T21:49:46-295a39b-170a72e/openbsd-386-64
2021-09-09T21:28:56-c163c31-c981874/openbsd-386-64
2021-09-09T19:43:39-c163c31-2c4f389/openbsd-386-64
2021-09-09T17:02:45-c163c31-f9271e4/openbsd-386-64
2021-09-09T16:48:19-c163c31-ea43445/openbsd-386-64
2021-09-09T15:34:08-c163c31-19457a5/openbsd-386-64
2021-09-09T15:27:16-c163c31-170a72e/openbsd-386-64
2021-09-09T15:07:50-c163c31-a295b3c/openbsd-386-64
2021-09-09T13:25:25-c163c31-66f0d35/openbsd-386-64
2021-09-09T13:25:25-c163c31-328cf2e/openbsd-386-64
2021-09-09T12:57:26-076821b-66f0d35/openbsd-386-64
2021-09-09T09:06:46-076821b-b86e8dd/openbsd-386-64
2021-09-09T04:47:41-076821b-c84f3a4/openbsd-386-64
2021-09-09T04:37:47-076821b-376a079/openbsd-386-64
2021-09-09T04:23:56-076821b-6edc579/openbsd-386-64
2021-09-09T01:25:40-076821b-d62866e/openbsd-386-64
2021-09-09T01:07:21-076821b-42563f8/openbsd-386-64
2021-09-09T00:29:35-076821b-4c52eac/openbsd-386-64
2021-09-09T00:10:46-076821b-e30a090/openbsd-386-64
2021-09-08T23:54:26-076821b-f5f8a91/openbsd-386-64
2021-09-08T23:54:26-076821b-328cf2e/openbsd-386-64
2021-09-08T23:35:19-2cae65c-f5f8a91/openbsd-386-64
2021-09-08T22:47:57-2cae65c-bff39cf/openbsd-386-64
2021-09-08T20:41:51-2cae65c-328cf2e/openbsd-386-64
2021-09-08T19:08:28-2cae65c-fcce86c/openbsd-386-64
2021-09-08T19:08:28-2cae65c-8226cb0/openbsd-386-64
2021-09-08T19:08:28-2cae65c-65f0d24/openbsd-386-64
2021-09-08T18:48:02-0a6f080-65f0d24/openbsd-386-64
2021-09-08T17:59:29-0a6f080-c8d4fe2/openbsd-386-64
2021-09-08T14:51:59-e5f719f-8226cb0/openbsd-386-64
2021-09-08T14:51:59-e5f719f-8214257/openbsd-386-64
2021-09-08T11:57:03-3604566-9295723/openbsd-386-64
2021-09-08T06:59:06-3604566-50c69cc/openbsd-386-64
2021-09-07T22:23:36-3604566-a9a01a3/openbsd-386-64
2021-09-07T20:37:05-3604566-8078355/openbsd-386-64
2021-09-07T19:14:23-3604566-903958d/openbsd-386-64
2021-09-07T17:50:42-3604566-8118866/openbsd-386-64
2021-09-07T17:38:14-3604566-2a48452/openbsd-386-64
2021-09-04T01:07:09-3604566-8226cb0/openbsd-386-64
2021-09-03T22:56:58-d39bbca-4957976/openbsd-386-64
2021-09-03T21:31:30-d39bbca-d9244f8/openbsd-386-64
2021-09-03T19:56:09-d39bbca-04d8d24/openbsd-386-64
2021-09-03T18:04:29-d39bbca-8226cb0/openbsd-386-64
2021-09-03T18:01:36-4e109c5-fcce86c/openbsd-386-64
2021-09-03T17:12:04-7391fc1-fcce86c/openbsd-386-64
2021-09-03T05:27:58-3b801c8-ab7c904/openbsd-386-64
2021-09-03T01:38:54-3b801c8-9f69a44/openbsd-386-64
2021-09-02T19:36:22-3b801c8-a8aa6cf/openbsd-386-64
2021-09-02T19:16:19-3b801c8-9633195/openbsd-386-64
2021-09-02T18:21:15-3b801c8-fcce86c/openbsd-386-64
2021-09-02T18:21:15-3b801c8-8226cb0/openbsd-386-64
2021-09-02T17:30:32-8373dc3-7609b50/openbsd-386-64
2021-09-02T17:00:30-8373dc3-acc2957/openbsd-386-64
2021-09-02T16:37:53-8373dc3-ead3fe0/openbsd-386-64
2021-09-02T16:15:00-8373dc3-d01388b/openbsd-386-64
2021-09-02T15:14:50-8373dc3-782aa42/openbsd-386-64
2021-09-02T15:05:39-8373dc3-d8b1567/openbsd-386-64
2021-09-02T14:44:50-8373dc3-014a972/openbsd-386-64
2021-09-02T14:13:09-8373dc3-8226cb0/openbsd-386-64
2021-09-02T07:31:53-8373dc3-840b429/openbsd-386-64
2021-09-02T06:44:04-8373dc3-1ae7ddc/openbsd-386-64
2021-09-01T22:54:04-8373dc3-fcce86c/openbsd-386-64
2021-09-01T16:41:15-8373dc3-1dd24ca/openbsd-386-64
2021-08-31T21:49:48-70fb962-1f83a8c/openbsd-386-64
2021-08-31T18:41:52-70fb962-4612130/openbsd-386-64
2021-08-31T17:42:36-70fb962-144e0b1/openbsd-386-64
2021-08-31T17:21:38-3e0d083-f27d6a2/openbsd-386-64
2021-08-31T16:43:46-3e0d083-6815235/openbsd-386-64
2021-08-31T16:20:54-3e0d083-605d1aa/openbsd-386-64
2021-08-31T16:14:01-3e0d083-ded10d7/openbsd-386-64
2021-08-31T08:41:16-3e0d083-f118d14/openbsd-386-64
2021-08-30T17:13:00-3e0d083-b602dae/openbsd-386-64
2021-08-30T16:00:10-3e0d083-7b38dd8/openbsd-386-64
2021-08-30T02:40:46-3e0d083-56c3856/openbsd-386-64
2021-08-25T15:30:19-c32dd46-099b819/openbsd-386-64
2021-08-25T03:13:06-c32dd46-e1fcf88/openbsd-386-64
2021-08-24T22:23:12-c32dd46-54cdef1/openbsd-386-64
2021-08-24T18:31:34-c32dd46-c2f96e6/openbsd-386-64
2021-08-24T15:55:38-c32dd46-d70c69d/openbsd-386-64
2021-08-24T14:51:57-c32dd46-ec51703/openbsd-386-64
2021-08-24T00:01:29-d4cc65f-8eeb1bf/openbsd-386-64
2021-08-23T22:55:34-d4cc65f-be1a693/openbsd-386-64
2021-08-23T20:32:04-d4cc65f-22540ab/openbsd-386-64
2021-08-23T20:04:34-d4cc65f-fa34678/openbsd-386-64
2021-08-23T19:49:23-d4cc65f-0a7f00a/openbsd-386-64
2021-08-23T17:29:25-d4cc65f-aeec6db/openbsd-386-64
2021-08-23T16:57:27-d4cc65f-f457ecc/openbsd-386-64
2021-08-23T15:27:42-d4cc65f-f1d8ea1/openbsd-386-64
2021-08-23T13:09:18-d4cc65f-4fbb5c8/openbsd-386-64
2021-08-22T23:49:55-d4cc65f-86ee892/openbsd-386-64
2021-08-20T21:27:50-d4cc65f-ec51703/openbsd-386-64
2021-08-20T21:00:08-bf6c7f2-97d17dc/openbsd-386-64
2021-08-20T19:58:13-bf6c7f2-f67e31d/openbsd-386-64
2021-08-20T19:44:02-bf6c7f2-ab9aaf4/openbsd-386-64
2021-08-20T18:55:57-bf6c7f2-5045477/openbsd-386-64
2021-08-20T18:45:25-bf6c7f2-0f25251/openbsd-386-64
2021-08-20T16:23:47-bf6c7f2-3034463/openbsd-386-64
2021-08-20T03:25:36-bf6c7f2-e9e0d1e/openbsd-386-64
2021-08-19T15:09:47-bf6c7f2-9871726/openbsd-386-64
2021-08-19T09:11:02-bf6c7f2-3bdc179/openbsd-386-64
2021-08-19T00:26:22-bf6c7f2-c85695a/openbsd-386-64
2021-08-18T17:22:40-bf6c7f2-ec51703/openbsd-386-64
2021-08-18T15:11:52-e32504c-0c83e01/openbsd-386-64
2021-08-17T23:59:53-e32504c-ddfcc02/openbsd-386-64
2021-08-17T16:22:15-96f147c-cf12b0d/openbsd-386-64
2021-08-17T15:00:04-96f147c-3001b0a/openbsd-386-64
2021-08-17T14:56:21-96f147c-0d53084/openbsd-386-64
2021-08-17T03:47:47-6932d22-d3deb2c/openbsd-386-64
2021-08-17T02:54:26-6932d22-ec51703/openbsd-386-64
2021-08-17T02:54:26-6932d22-29ec74f/openbsd-386-64
2021-08-17T01:49:12-10bcabd-a8d39f1/openbsd-386-64
2021-08-16T23:57:10-10bcabd-2a19333/openbsd-386-64
2021-08-16T23:03:03-10bcabd-df9c5d8/openbsd-386-64
2021-08-16T21:38:38-10bcabd-c04a32e/openbsd-386-64
2021-08-16T20:51:50-10bcabd-ec51703/openbsd-386-64
2021-08-16T20:23:26-758a1a1-a05a7d4/openbsd-386-64
2021-08-16T20:02:13-758a1a1-ec27168/openbsd-386-64
2021-08-16T18:52:20-758a1a1-d35035f/openbsd-386-64
2021-08-16T17:24:37-758a1a1-850768b/openbsd-386-64
2021-08-16T16:58:24-758a1a1-a0adf91/openbsd-386-64
2021-08-16T16:11:44-758a1a1-5da2010/openbsd-386-64
2021-08-16T15:27:42-a55d515-5da2010/openbsd-386-64
2021-08-16T13:39:19-a55d515-7b7d7d7/openbsd-386-64
2021-08-16T12:54:44-a55d515-c88e3ff/openbsd-386-64
2021-08-16T08:37:46-a55d515-fcdc3c0/openbsd-386-64
2021-08-16T07:44:47-a55d515-160d797/openbsd-386-64
2021-08-15T19:08:56-a55d515-6ed9463/openbsd-386-64
2021-08-15T15:51:49-a55d515-48dfddb/openbsd-386-64
2021-08-15T02:18:46-45389f5-48dfddb/openbsd-386-64
2021-08-14T22:44:35-45389f5-ff3469b/openbsd-386-64
2021-08-13T16:57:31-45389f5-acbe242/openbsd-386-64
2021-08-13T03:36:45-8fae06a-d66b211/openbsd-386-64
2021-08-13T03:36:45-8fae06a-4c8ffb3/openbsd-386-64
2021-08-13T03:35:06-3fce476-4c8ffb3/openbsd-386-64
2021-08-13T02:31:39-03a91dd-d66b211/openbsd-386-64
2021-08-12T21:33:09-d52cb71-0d01934/openbsd-386-64
2021-08-12T21:18:49-d52cb71-acbe242/openbsd-386-64
2021-08-12T21:18:49-d52cb71-46fd547/openbsd-386-64
2021-08-12T18:45:58-9f49561-5805efc/openbsd-386-64
2021-08-12T17:14:32-15a2ab8-095bb79/openbsd-386-64
2021-08-12T16:12:36-749a5cd-acbe242/openbsd-386-64
2021-08-12T15:58:41-f91c4a5-095bb79/openbsd-386-64
2021-08-11T22:07:50-f91c4a5-dea23e9/openbsd-386-64
2021-08-11T16:14:09-5f5a173-d66b211/openbsd-386-64
2021-08-11T15:16:17-5f5a173-acbe242/openbsd-386-64
2021-08-11T15:16:17-5f5a173-1f9c9d8/openbsd-386-64
2021-08-11T04:51:32-92b2fbe-1f9c9d8/openbsd-386-64
2021-08-09T22:50:32-337cebd-acbe242/openbsd-386-64
2021-08-09T20:21:32-fcc905b-fa6aa87/openbsd-386-64
2021-08-09T20:21:32-fcc905b-f1dce31/openbsd-386-64
2021-08-09T20:21:06-bfe69c3-acbe242/openbsd-386-64
2021-08-09T20:20:24-0d28b7d-fa6aa87/openbsd-386-64
2021-08-09T20:20:24-0d28b7d-acbe242/openbsd-386-64
2021-08-09T20:06:35-4869184-f1dce31/openbsd-386-64
2021-08-09T17:59:45-4869184-fa6aa87/openbsd-386-64
2021-08-09T17:59:45-4869184-7aeaad5/openbsd-386-64
2021-08-07T01:29:22-d529aec-891547e/openbsd-386-64
2021-08-06T20:44:03-d529aec-63b968f/openbsd-386-64
2021-08-06T16:51:12-f367f01-70546f6/openbsd-386-64
2021-08-05T20:22:31-f367f01-fd45e26/openbsd-386-64
2021-08-04T20:44:12-f68a40b-fa6aa87/openbsd-386-64
2021-08-04T20:44:05-f68a40b-acbe242/openbsd-386-64
2021-08-04T17:02:12-f68a40b-e90f402/openbsd-386-64
2021-08-04T17:02:12-f68a40b-6e73886/openbsd-386-64
2021-08-04T15:26:45-309db04-6e73886/openbsd-386-64
2021-08-04T13:50:16-309db04-8a7ee4c/openbsd-386-64
2021-08-03T20:45:05-2f64839-8a7ee4c/openbsd-386-64
2021-08-03T19:59:22-32c652e-7921829/openbsd-386-64
2021-08-03T19:59:22-32c652e-16ab7e4/openbsd-386-64
2021-08-03T17:18:10-7a2ec09-8a7ee4c/openbsd-386-64
2021-08-03T16:06:16-594b3a2-8a7ee4c/openbsd-386-64
2021-08-03T15:39:08-3228160-8a7ee4c/openbsd-386-64
2021-08-03T15:00:55-3395cb0-1b68e07/openbsd-386-64
2021-08-02T22:26:52-3395cb0-3d5afa9/openbsd-386-64
2021-08-02T22:23:30-3395cb0-0fb1e14/openbsd-386-64
2021-08-02T22:22:46-3395cb0-ba93baa/openbsd-386-64
2021-08-02T21:22:11-3395cb0-8a7ee4c/openbsd-386-64
2021-08-02T20:18:28-bb69444-accf363/openbsd-386-64
2021-08-02T20:18:07-f8cfada-c6d89db/openbsd-386-64
2021-08-02T20:18:07-f8cfada-accf363/openbsd-386-64
2021-08-02T20:18:07-f8cfada-8a7ee4c/openbsd-386-64
2021-08-02T20:16:24-45eff0f-accf363/openbsd-386-64
2021-08-02T20:16:24-45eff0f-8a7ee4c/openbsd-386-64
2021-08-02T19:06:04-a668498-c6d89db/openbsd-386-64
2021-08-02T17:18:57-3810fa8-8a7ee4c/openbsd-386-64
2021-07-31T00:26:46-3810fa8-c6d89db/openbsd-386-64
2021-07-29T15:30:38-ab1fe72-70fd4e4/openbsd-386-64
2021-07-27T22:01:54-07bc1bf-7cd10c1/openbsd-386-64
2021-07-27T00:15:55-07bc1bf-33ff155/openbsd-386-64
2021-07-26T20:36:31-07bc1bf-ecaa681/openbsd-386-64
2021-07-26T20:19:17-4ad98e9-ed8cbbc/openbsd-386-64
2021-07-26T17:47:47-ebce39e-ecaa681/openbsd-386-64
2021-07-26T16:44:13-4fe0d6c-1868f82/openbsd-386-64
2021-07-26T16:25:21-0c506a2-1868f82/openbsd-386-64
2021-07-23T18:08:39-46d1522-052da57/openbsd-386-64
2021-07-22T21:53:16-c740bfd-ed8cbbc/openbsd-386-64
2021-07-22T17:13:07-251092d-fdb45ac/openbsd-386-64
2021-07-21T20:52:30-7f68387-c6d89db/openbsd-386-64
2021-07-21T20:52:30-7f68387-3e48c03/openbsd-386-64
2021-07-21T20:29:18-7aa8294-3e48c03/openbsd-386-64
2021-07-21T16:31:48-7aa8294-c6d89db/openbsd-386-64
2021-07-21T16:31:48-7aa8294-bc51e93/openbsd-386-64
2021-07-19T18:00:46-6e9046b-404127c/openbsd-386-64
2021-07-19T17:27:58-6e9046b-6298cfe/openbsd-386-64
2021-07-18T21:03:43-6e9046b-a66190e/openbsd-386-64
2021-07-16T02:43:48-6e9046b-aa4e0f5/openbsd-386-64
2021-07-15T20:16:41-6e9046b-69728ea/openbsd-386-64
2021-07-15T17:35:20-6e9046b-c1cc9f9/openbsd-386-64
2021-07-15T13:35:41-6e9046b-c6d89db/openbsd-386-64
2021-07-15T00:09:22-0cf4e27-21a04e3/openbsd-386-64
2021-07-14T22:52:15-0cf4e27-2b00a54/openbsd-386-64
2021-07-14T22:42:09-0cf4e27-60ddf42/openbsd-386-64
2021-07-14T17:25:06-5061c41-60ddf42/openbsd-386-64
2021-07-13T20:15:39-ef97713-a985897/openbsd-386-64
2021-07-13T16:40:23-8e85a28-a985897/openbsd-386-64
2021-07-13T13:36:40-d36a54b-bc51e93/openbsd-386-64
2021-07-13T13:36:40-d36a54b-a985897/openbsd-386-64
2021-07-13T12:52:46-3844600-c6d89db/openbsd-386-64
2021-07-12T22:00:43-de44776-c6d89db/openbsd-386-64
2021-07-12T22:00:43-de44776-bc51e93/openbsd-386-64
2021-07-12T20:58:00-a7dfe3d-a985897/openbsd-386-64
2021-07-12T17:19:11-a7dfe3d-c77980b/openbsd-386-64
2021-07-12T16:18:15-a7dfe3d-fb052db/openbsd-386-64
2021-07-10T00:41:40-cb1acef-fb052db/openbsd-386-64
2021-07-10T00:41:40-cb1acef-3d1d066/openbsd-386-64
2021-07-09T18:00:16-8e32e9f-ab4085c/openbsd-386-64
2021-07-09T17:21:04-8e32e9f-296ddf2/openbsd-386-64
2021-07-08T23:16:08-6994825-fb052db/openbsd-386-64
2021-07-08T21:10:20-e688b94-fb052db/openbsd-386-64
2021-07-08T19:58:58-2f04284-296ddf2/openbsd-386-64
2021-07-08T19:58:23-41a65bd-296ddf2/openbsd-386-64
2021-07-08T19:56:33-febfa9d-3d1d066/openbsd-386-64
2021-07-08T19:56:33-febfa9d-296ddf2/openbsd-386-64
2021-07-08T19:56:07-71eae3a-296ddf2/openbsd-386-64
2021-07-08T19:20:05-55cd480-3d1d066/openbsd-386-64
2021-07-08T17:53:43-55cd480-296ddf2/openbsd-386-64
2021-07-08T16:22:39-55cd480-2ca44fe/openbsd-386-64
2021-07-08T16:04:41-77c1b7a-b51bf4f/openbsd-386-64
2021-07-08T16:04:41-77c1b7a-2ca44fe/openbsd-386-64
2021-07-08T02:47:23-640c1de-5c59e11/openbsd-386-64
2021-07-07T20:06:16-fd00574-c96833e/openbsd-386-64
2021-07-02T20:11:05-7edcfe5-287c5e8/openbsd-386-64
2021-07-01T08:39:23-20dafe5-cb4cd9e/openbsd-386-64
2021-07-01T08:39:23-20dafe5-b51bf4f/openbsd-386-64
2021-07-01T08:39:23-20dafe5-9d65578/openbsd-386-64
2021-06-30T22:02:09-f0847e0-4711bf3/openbsd-386-64
2021-06-30T18:03:06-f0847e0-c080d03/openbsd-386-64
2021-06-30T16:45:04-f0847e0-b51bf4f/openbsd-386-64
2021-06-30T02:00:49-72e4d1b-d19a533/openbsd-386-64
2021-06-30T01:29:49-72e4d1b-c45e800/openbsd-386-64
2021-06-29T19:12:30-72e4d1b-b51bf4f/openbsd-386-64
2021-06-29T17:38:19-100b229-b51bf4f/openbsd-386-64
2021-06-29T14:11:33-100b229-fd4b587/openbsd-386-64
2021-06-29T02:34:20-12f8456-e2e05af/openbsd-386-64
2021-06-27T05:06:30-12f8456-c95464f/openbsd-386-64
2021-06-25T15:37:24-636017e-fd129a6/openbsd-386-64
2021-06-25T15:05:05-48cad5e-37f9a8f/openbsd-386-64
2021-06-24T03:45:33-bfc1674-a9bb382/openbsd-386-64
2021-06-24T03:06:00-bfc1674-44a12e5/openbsd-386-64
2021-06-22T19:12:15-6d3e439-fd129a6/openbsd-386-64
2021-06-22T16:59:10-4c651fc-0ebd5a8/openbsd-386-64
2021-06-22T16:07:17-4c651fc-5bd09e5/openbsd-386-64
2021-06-22T11:31:57-d25f906-63daa77/openbsd-386-64
2021-06-21T22:46:57-d25f906-fd129a6/openbsd-386-64
2021-06-21T21:21:09-d25f906-1bd5a20/openbsd-386-64
2021-06-21T20:53:11-d25f906-761edf7/openbsd-386-64
2021-06-21T16:58:36-d25f906-16e82be/openbsd-386-64
2021-06-18T23:57:09-d25f906-86743e7/openbsd-386-64
2021-06-18T18:34:00-d25f906-57aaa19/openbsd-386-64
2021-06-15T18:42:11-4b484fb-219fe9d/openbsd-386-64
2021-06-15T17:05:31-4b484fb-033d885/openbsd-386-64
2021-06-15T10:13:08-4b484fb-abc56fd/openbsd-386-64
2021-06-14T22:42:48-4b484fb-7841cb1/openbsd-386-64
2021-06-14T19:26:24-4b484fb-d4f34f8/openbsd-386-64
2021-06-11T16:06:07-9a55cb1-2f11284/openbsd-386-64
2021-06-10T18:32:07-9aa007e-dc00dc6/openbsd-386-64
2021-06-10T17:33:43-f6327c5-dc00dc6/openbsd-386-64
2021-06-10T17:33:43-f6327c5-ab7f829/openbsd-386-64
2021-06-09T21:40:20-13cf844-7677616/openbsd-386-64
2021-06-09T15:58:30-e0b9cf7-e6dda19/openbsd-386-64
2021-06-09T15:58:30-e0b9cf7-ab7f829/openbsd-386-64
2021-06-09T15:58:24-16e5f55-e6dda19/openbsd-386-64
2021-06-09T14:15:55-890984b-ab7f829/openbsd-386-64
2021-06-08T16:06:52-4e58f8f-2169deb/openbsd-386-64
2021-06-07T17:02:02-234f954-8212707/openbsd-386-64
2021-06-07T15:22:48-126df1d-ab7f829/openbsd-386-64
2021-06-07T15:22:48-126df1d-7677616/openbsd-386-64
2021-06-05T19:52:26-1225b6f-e1fa260/openbsd-386-64
2021-06-05T09:28:43-1225b6f-e3cb381/openbsd-386-64
2021-06-04T20:51:27-1225b6f-95939e8/openbsd-386-64
2021-06-04T20:34:05-1225b6f-ab7f829/openbsd-386-64
2021-06-04T17:45:44-7295a4e-ab7f829/openbsd-386-64
2021-06-04T17:45:44-7295a4e-831f937/openbsd-386-64
2021-06-04T09:39:32-bf13205-0214440/openbsd-386-64
2021-06-04T01:02:31-bf13205-7677616/openbsd-386-64
2021-06-03T19:38:56-8f2cf6c-6d98301/openbsd-386-64
2021-06-03T17:14:37-4abb1e2-ab7f829/openbsd-386-64
2021-06-03T16:41:39-4abb1e2-e0d029f/openbsd-386-64
2021-06-03T16:41:39-4abb1e2-0d6115c/openbsd-386-64
2021-06-03T03:03:28-1c2154a-6b411e9/openbsd-386-64
2021-06-02T21:39:28-384c392-dd7ba3b/openbsd-386-64
2021-06-02T19:45:53-384c392-4f572d7/openbsd-386-64
2021-06-02T19:26:46-29f5b8f-f2222d8/openbsd-386-64
2021-06-02T19:23:16-7922491-fbf844b/openbsd-386-64
2021-06-02T18:55:23-726034e-dbf69b7/openbsd-386-64
2021-06-02T18:55:23-726034e-448be06/openbsd-386-64
2021-06-02T16:25:20-7271753-6e189af/openbsd-386-64
2021-06-02T01:35:15-377464f-e3c9537/openbsd-386-64
2021-06-01T22:58:04-377464f-0410005/openbsd-386-64
2021-06-01T17:08:12-df07577-2725522/openbsd-386-64

Loading

@bcmills
Copy link
Member

@bcmills bcmills commented Nov 2, 2021

The openbsd-amd64 involvement may be new, though:

greplogs --dashboard -md -l -e '(?ms)\Aopenbsd-amd64.*fatal error:' --since=2021-06-01

2021-11-02T00:12:17-4ff95d6-81fea0b/openbsd-amd64-68
2021-11-01T02:47:30-fde4cc2/openbsd-amd64-68
2021-10-28T03:44:12-2ff1074/openbsd-amd64-68
2021-10-27T18:08:49-08f2e89-bb49eb3/openbsd-amd64-68

Loading

@mknyszek
Copy link
Contributor Author

@mknyszek mknyszek commented Nov 2, 2021

Yeah, I think the amd64 one is the new thing. It's also higher priority since that's a trybot and it's causing trybot flakes.

Loading

@bcmills
Copy link
Member

@bcmills bcmills commented Nov 3, 2021

Loading

@bcmills
Copy link
Member

@bcmills bcmills commented Nov 3, 2021

This is a release blocker for 1.18 in that we need to figure out what to do about it.

OpenBSD is not a first-class port, so this does not necessarily block the 1.18 release — but if we are intentionally releasing 1.18 with unstable OpenBSD support its status should be documented in the release notes.

Loading

@ALTree
Copy link
Member

@ALTree ALTree commented Nov 6, 2021

trybot error (not sure if related): https://storage.googleapis.com/go-build-log/25f64550/openbsd-amd64-68_09e5b260.log

fatal error: bad g->status in ready

Loading

@mknyszek
Copy link
Contributor Author

@mknyszek mknyszek commented Nov 9, 2021

Spinning up gomotes for this too. Going to attempt at getting a core dump.

Loading

@mknyszek
Copy link
Contributor Author

@mknyszek mknyszek commented Nov 10, 2021

Took about 4 hours and 15 minutes of 10 gomotes spinning, but I reproduced it. No core file. :( I'd poke around the gomote but unfortunately I got to it too late. I'll try again tomorrow.

Loading

@mknyszek
Copy link
Contributor Author

@mknyszek mknyszek commented Nov 11, 2021

Wow. I've been spinning in 10 gomotes for 22h without a crash.

Loading

@bcmills
Copy link
Member

@bcmills bcmills commented Nov 11, 2021

Loading

@bcmills
Copy link
Member

@bcmills bcmills commented Nov 11, 2021

(Filed clock_gettime_trampoline separately as #49532, since it looks like it may be an older bug.)

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants