New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: reverse buildlet doesn't limit all.bash run time? #10921

Open
minux opened this Issue May 21, 2015 · 2 comments

Comments

Projects
None yet
4 participants
@minux
Member

minux commented May 21, 2015

http://farmer.golang.org/logs?name=darwin-arm-a5ios&rev=8fa14ea8b4744576bd28073901154c15813e29de&st=0xc20847ca80

  builder: darwin-arm-a5ios
      rev: 8fa14ea8b4744576bd28073901154c15813e29de
 buildlet: http://none
  started: 2015-05-16 02:02:58.850844883 +0000 UTC
   status: still running

Events:
         2015-05-16T02:02:58Z get_buildlet
   +0.0s 2015-05-16T02:02:58Z waiting_machine_in_use
 +406284.8s 2015-05-20T18:54:23Z got_machine
   +0.5s 2015-05-20T18:54:24Z cleaned_up
   +0.0s 2015-05-20T18:54:24Z got_buildlet
   +0.0s 2015-05-20T18:54:24Z start_write_version_tar
   +0.0s 2015-05-20T18:54:24Z start_fetch_gerrit_tgz
   +0.2s 2015-05-20T18:54:24Z start_write_go14_tar
   +0.0s 2015-05-20T18:54:24Z start_write_go_tar
   +2.5s 2015-05-20T18:54:26Z end_write_go14_tar
   +1.6s 2015-05-20T18:54:28Z end_write_go_tar
   +0.0s 2015-05-20T18:54:28Z pre_exec
   +0.0s 2015-05-20T18:54:28Z running_exec

FAIL    hash/adler32    239.226s
 process continue
Process 493 resuming
go_darwin_arm_exec: timeout running tests
FAIL    hash/crc32  239.234s
 process continue
Process 499 resuming
go_darwin_arm_exec: timeout running tests
FAIL    hash/crc64  239.225s
 process continue
Process 504 resuming
go_darwin_arm_exec: timeout running tests
FAIL    hash/fnv    239.228s
 process continue
Process 509 resuming
go_darwin_arm_exec: timeout running tests
FAIL    html    239.227s
 process continue
Process 515 resuming
go_darwin_arm_exec: timeout running tests
FAIL    html/template   239.280s
 process continue
Process 520 resuming
go_darwin_arm_exec: timeout running tests
FAIL    image   239.266s
 process continue
Process 525 resuming
go_darwin_arm_exec: timeout running tests
FAIL    image/color 239.259s
?       image/color/palette [no test files]
 process continue
Process 531 resuming
go_darwin_arm_exec: timeout running tests
FAIL    image/draw  239.268s
 process continue
Process 536 resuming
go_darwin_arm_exec: timeout running tests
FAIL    image/gif   239.266s
?       image/internal/imageutil    [no test files]
 process continue
Process 541 resuming
go_darwin_arm_exec: timeout running tests
FAIL    image/jpeg  239.274s
....

The build has been running for more than 5 hours, and waited more than 4 days
to get a machine. Clearly the timeout limit logic is missing or doesn't work.

/cc @crawshaw

@minux minux added this to the Unreleased milestone May 21, 2015

@rsc rsc added the Builders label Jun 11, 2015

@bradfitz bradfitz changed the title from x/build: reverse builtlet doesn't limit all.bash run time? to x/build: reverse buildlet doesn't limit all.bash run time? Jul 7, 2015

@josharian

This comment has been minimized.

Contributor

josharian commented Aug 10, 2015

The dev.ssa 386 and arm builders have been hung in a CPU-burning infinite loop for three or four days.

I'll mail a fix shortly, but a global timeout would be really good.

@bradfitz

This comment has been minimized.

Member

bradfitz commented Aug 11, 2015

They don't run all.bash. They run make.bash and then go tool dist test --list and then run go tool dist test <name> for each.

But yeah, perhaps a timeout is missing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment