Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: frequent timeouts running js-wasm TryBots #35170

Open
bcmills opened this issue Oct 26, 2019 · 4 comments

Comments

@bcmills
Copy link
Member

@bcmills bcmills commented Oct 26, 2019

I've noticed that the js-wasm TryBots on my recent CLs seem to start with one or more spurious failures before finally running successfully.

I suspect that that increases load on the TryBots generally, and may increase overall TryBot latency in some circumstances.

A log from a recent example is here (https://farmer.golang.org/temporarylogs?name=js-wasm&rev=9516a47489b92b64e9cf5633e3ab7e37def3353f&st=0xc00ab39b80):

  builder: js-wasm
      rev: 9516a47489b92b64e9cf5633e3ab7e37def3353f
 buildlet: (nil *buildlet.Client)
  started: 2019-10-26 01:37:51.137407213 +0000 UTC m=+287652.939853079
    ended: 2019-10-26 01:38:21.20458221 +0000 UTC m=+287683.007028020
  success: false

Events:
  2019-10-26T01:37:51Z ask_maintner_has_ancestor 
  2019-10-26T01:38:21Z finish_ask_maintner_has_ancestor after 30s; err=context deadline exceeded

Build log:
Error checking whether commit 9516a47489b92b64e9cf5633e3ab7e37def3353f includes ancestor 3dced519cbabc213df369d9112206986e62687fa: context deadline exceeded


Error: context deadline exceeded

CC @dmitshur @toothrot @bradfitz

@agnivade

This comment has been minimized.

Copy link
Contributor

@agnivade agnivade commented Oct 26, 2019

One change was that the builder was recently bumped to Node 13.

@bradfitz

This comment has been minimized.

Copy link
Member

@bradfitz bradfitz commented Oct 27, 2019

I think that's unrelated. When I've seen this, the error is also as Bryan quoted above: a timeout doing a gRPC call to maintner, asking whether we should even do this build. We used to need to skip builds for js/wasm if the git history of the rev to be built didn't include js-wasm support. But now that we've supported js/wasm for a number of releases (since Go 1.11) and we don't even support Go 1.10 (or even Go 1.11) any more, we can just remove that condition on that builder.

But also: maintner shouldn't be so slow at that query.

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Oct 27, 2019

Change https://golang.org/cl/203757 mentions this issue: dashboard: stop checking whether the branch/rev supports js/wasm for js/wasm

gopherbot pushed a commit to golang/build that referenced this issue Oct 28, 2019
…js/wasm

It's supported it for ages. We no longer support or build branches
that didn't have js/wasm support.

See golang/go#35170 (comment)

Updates golang/go#35170 (still need a maintner fix, ideally)

Change-Id: If5489dd266dea9a3a48bbc29f83528c0f939d23b
Reviewed-on: https://go-review.googlesource.com/c/build/+/203757
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@bradfitz

This comment has been minimized.

Copy link
Member

@bradfitz bradfitz commented Oct 28, 2019

Not very scientific, but the has-ancestor RPCs are taking only 100ms for me. (includes TCP+TLS setup, etc) Not anywhere near 30 seconds.

$ time maintq has-ancestor 0ae9389609f23dc905c58fc2ad7bcc16b770f337 3dced519cbabc213df369d9112206986e62687fa
has_ancestor:true

real    0m0.105s
user    0m0.076s
sys     0m0.028s
codebien added a commit to codebien/build that referenced this issue Nov 13, 2019
…js/wasm

It's supported it for ages. We no longer support or build branches
that didn't have js/wasm support.

See golang/go#35170 (comment)

Updates golang/go#35170 (still need a maintner fix, ideally)

Change-Id: If5489dd266dea9a3a48bbc29f83528c0f939d23b
Reviewed-on: https://go-review.googlesource.com/c/build/+/203757
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.