Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: linux-mips64 builder not reporting success or failure #19953

Closed
bradfitz opened this issue Apr 12, 2017 · 3 comments

Comments

Projects
None yet
3 participants
@bradfitz
Copy link
Member

commented Apr 12, 2017

(Continued from #19930 (comment))

The linux-mips64 builder keeps working, but no success or failure shows up on build.golang.org, which means the builder is disappearing or something.

I see make.bash finishes in 13 minutes, and now I'm watching it do the snapshot copy to Google Cloud Storage. There's a comment on that code that says:

func (st *buildStatus) writeSnapshot(bc *buildlet.Client) (err error) {
        sp := st.createSpan("write_snapshot_to_gcs")
        defer func() { sp.done(err) }()
        // This should happen in 15 seconds or so, but I saw timeouts                                                                                                          
        // a couple times at 1 minute. Some buildlets might be far                                                                                                             
        // away on the network, so be more lenient. The timeout mostly                                                                                                         
        // is here to prevent infinite hangs.                                                                                                                                  
        ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
        defer cancel()

There's a 5 minute timeout. It's possible those arm5 machines can't gzip quickly, and/or their network is slow.

Yup, confirmed:

Error: writeSnapshot: net/http: request canceled

Maybe we don't need to snapshot those builds to Google Cloud Storage. That means we can't shard tests, but we're not doing that currently anyway.

/cc @vstefanovic @adams-sarah @cybrcodr

@gopherbot

This comment has been minimized.

Copy link

commented Apr 13, 2017

CL https://golang.org/cl/40505 mentions this issue.

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Apr 13, 2017

mips64 has an "ok" on the dashboard now.

@vstefanovic

This comment has been minimized.

Copy link
Member

commented Apr 13, 2017

@bradfitz, thanks!

(mips and mips64 builders are slow partly because they run on the same machine, they are slowing down each other.)

@golang golang locked and limited conversation to collaborators Apr 13, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.