New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os/exec: TestExtraFiles is flaky on linux-386-sid builder #25628

Open
ALTree opened this Issue May 29, 2018 · 9 comments

Comments

Projects
None yet
6 participants
@ALTree
Member

ALTree commented May 29, 2018

I've seen several of these:

--- FAIL: TestExtraFiles (0.10s)
	exec_test.go:607: Run: exit status 1; stdout "leaked parent file. fd = 66; want 65\n", stderr ""
FAIL
FAIL	os/exec	0.687s

on the linux-386-sid builder in the past weeks. Here's one:

https://build.golang.org/log/b95e0220b4130af6c1bcfed59ec934568717d0ef

@bcmills bcmills added the OS-Linux label May 29, 2018

@ianlancetaylor ianlancetaylor added this to the Go1.11 milestone May 31, 2018

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented May 31, 2018

That is odd. The errors show relatively large descriptor numbers. It implies that something in the system is opening a large descriptor before running make.bash.

@bcmills

This comment has been minimized.

Member

bcmills commented Jun 1, 2018

There is another recurring failure mode in os/exec on linux-386-sid that produces broken pipe instead. Any idea whether they're related?

An example failure (https://build.golang.org/log/cef4b584d2ccb30621e2965de0517f6077aa2f92):

testing: can't write /tmp/go-build619681385/b056/testlog.txt: write /tmp/go-build619681385/b056/testlog.txt: broken pipe
FAIL	os/exec	0.697s
@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Jun 1, 2018

Yes, they could be related. The os/exec package tries to guess which descriptor is being used for testlog.txt; if it guesses wrong, because of an unexpectedly open descriptor, that could cause the broken pipe error.

@ALTree ALTree changed the title from os/exec: TestExtraFiles is flaky on linux-386-sid bulder to os/exec: TestExtraFiles is flaky on linux-386-sid builder Jun 2, 2018

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Jun 27, 2018

2018-05-09T00:05:52-c1492b6/linux-386-sid
2018-05-11T14:40:34-e405c9c/linux-386-sid
2018-05-11T23:11:53-e6a9335/linux-386-sid
2018-05-12T06:21:01-f94b5a8/linux-386-sid
2018-05-14T13:45:18-c06f027/linux-386-sid
2018-05-14T14:12:31-7bac2a9/linux-386-sid
2018-05-14T17:59:59-15f2cbf/linux-386-sid
2018-05-15T16:29:04-212c947/linux-386-sid
2018-05-16T12:01:16-4102e6f/linux-386-sid
2018-05-21T13:18:16-f2cde55/linux-386-sid
2018-05-22T14:15:44-b812eec/linux-386-sid
2018-05-22T15:55:01-840f25b/linux-386-sid
2018-05-22T18:47:03-3d15f76/linux-386-sid
2018-05-22T18:49:31-f045ddc/linux-386-sid
2018-05-22T20:27:52-402dd10/linux-386-sid
2018-05-22T20:44:03-3c36b8b/linux-386-sid
2018-05-23T01:45:00-92bdfab/linux-386-sid
2018-05-23T14:13:24-ef880a2/linux-386-sid
2018-05-23T19:01:58-3f89214/linux-386-sid
2018-05-24T17:19:09-8aa57f1/linux-386-sid
2018-05-24T18:03:21-b0b0921/linux-386-sid
2018-05-24T20:26:47-9eb2c43/linux-386-sid
2018-05-24T22:08:00-30b6bc3/linux-386-sid
2018-05-25T08:42:09-e52c1c0/linux-386-sid
2018-05-25T13:09:29-bdaadeb/linux-386-sid
2018-05-27T07:33:08-c1d9d1f/linux-386-sid
2018-05-29T02:39:16-002c764/linux-386-sid
2018-05-29T17:46:19-ef53de8/linux-386-sid
2018-05-29T20:27:03-4790181/linux-386-sid
2018-05-29T21:54:57-210a9e0/linux-386-sid
2018-05-30T09:10:04-7d704a9/linux-386-sid
2018-05-30T13:40:36-e023d5b/linux-386-sid
2018-05-31T13:29:50-caf9686/linux-386-sid
2018-05-31T15:13:02-08e2e88/linux-386-sid
2018-05-31T17:29:10-424c215/linux-386-sid
2018-06-01T17:36:40-6cfeedb/linux-386-sid
2018-06-04T18:33:41-d6c3b0a/linux-386-sid
2018-06-05T00:29:50-75c1aed/linux-386-sid
2018-06-06T17:59:08-11f6cb4/linux-386-sid
2018-06-06T19:20:12-ca8ec69/linux-386-sid
2018-06-07T07:00:46-e5f0c1f/linux-386-sid
2018-06-07T14:31:33-35778aa/linux-386-sid
2018-06-07T15:47:29-0e1517c/linux-386-sid
2018-06-07T16:17:04-7b08e61/linux-386-sid
2018-06-08T22:44:03-77d5c62/linux-386-sid
2018-06-10T05:49:55-a5f8303/linux-386-sid
2018-06-11T12:13:11-30a63ec/linux-386-sid
2018-06-11T17:45:36-9e9ff56/linux-386-sid
2018-06-11T20:58:09-92f8acd/linux-386-sid
2018-06-12T17:05:39-ec98933/linux-386-sid
2018-06-12T20:22:50-44b826b/linux-386-sid
2018-06-12T20:37:02-bd83774/linux-386-sid
2018-06-13T17:55:32-a1b85ee/linux-386-sid
2018-06-13T20:32:08-fb4fb04/linux-386-sid
2018-06-13T21:43:06-1e721cf/linux-386-sid
2018-06-13T21:57:41-fc0e1d2/linux-386-sid
2018-06-13T22:01:28-efddc16/linux-386-sid
2018-06-14T13:21:46-1031380/linux-386-sid
2018-06-15T02:01:04-8998c55/linux-386-sid
2018-06-21T22:55:55-4b34289/linux-386-sid
2018-06-22T14:27:22-00eac89/linux-386-sid

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Jun 27, 2018

Since this doesn't seem to be happening anywhere else, this is most likely a problem specific to the linux-386-sid builder. Something on that system is causing the os/exec tests to run with an unexpectedly open file descriptor.

@bradfitz Is there anything unusual about the linux-386-sid builder?

@ianlancetaylor ianlancetaylor modified the milestones: Go1.11, Go1.12 Jun 27, 2018

@bradfitz

This comment has been minimized.

Member

bradfitz commented Jun 27, 2018

The 386-sid builder is an amd64 Docker container (like all our linux-386-*) builders with the "debian:sid" base layer (https://github.com/golang/build/blob/master/env/linux-x86-sid/Dockerfile). We occasionally re-build it to reflect the ever-changing Debian sid.

Last updates:

  • April 11, 2017
  • Nov 2, 2017
  • May 8, 2018 (current)
@ALTree

This comment has been minimized.

Member

ALTree commented Jul 7, 2018

This was reported to also happen on a linux_386 Virtualbox machine with an updated Ubuntu Mate 18.04 LTS and boostrap Go version 1.4.3 (Issue #26261).

@mikesmitty

This comment has been minimized.

mikesmitty commented Aug 24, 2018

Ran into this issue with 1.11rc2 on Fedora 27 i386 as well:

--- FAIL: TestExtraFiles (0.07s)
    exec_test.go:607: Run: exit status 1; stdout "leaked parent file. fd = 66; want 65\n", stderr ""
FAIL
FAIL	os/exec	1.074s
@bobvanderlinden

This comment has been minimized.

bobvanderlinden commented Sep 15, 2018

This happens on NixOS as well. There it seems to be happening reliably as I can reproduce it on my laptop again and again. The NixOS build server also reproduces the error for every build.

This happens on go 1.11.

For reference: https://hydra.nixos.org/build/80999166/nixlog/12

bobvanderlinden added a commit to bobvanderlinden/nixpkgs that referenced this issue Sep 15, 2018

go: skip flaky os/exec TestExtraFiles
This seems to only happen on i686 on go 1.11. It also happens on
Debian. For more information:

golang/go#25628

@bobvanderlinden bobvanderlinden referenced this issue Sep 15, 2018

Merged

go: skip flaky os/exec TestExtraFiles #46706

4 of 9 tasks complete

Mic92 added a commit to NixOS/nixpkgs that referenced this issue Sep 16, 2018

go: skip flaky os/exec TestExtraFiles
This seems to only happen on i686 on go 1.11. It also happens on
Debian. For more information:

golang/go#25628
(cherry picked from commit 0bf6b44)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment