x/build: 3 of 6 corellium iOS and Android builders are missing

[dmitshur]

From https://farmer.golang.org#pools:

• host-android-arm64-corellium-android: 0/0 (3 missing)
• host-darwin-arm64-corellium-ios: 0/0 (3 missing)

Their results have stopped showing up on https://build.golang.org since around Jul 17.

/cc @znly per builder owners. Are you able to take a look at this? Thanks.

/cc @cagedmantis @toothrot @andybons

[dmitshur]

These builders were added in #31722. Perhaps @znly isn't the best GitHub account to ping about this, because it's an organization.

/cc @eliasnaur @bradfitz @steeve Do you know who would be a good person to ping to look into this issue? Thank you.

[eliasnaur]

All the builders were turned off at Corellium. Perhaps someone on the Go team would like access to the Corellium dashboard for when it happens in the future? I'm not actively checking the build.golang.org dashboard.

[steeve]

Let me know if you need to add people to the org.

On a side note, I see the results are back.

[dmitshur]

Thank you for looking at this issue. I see that some of the builders are back, although it seems to be 3 out of 6 in total so far:

host-android-arm64-corellium-android: 1/1 (2 missing)
host-darwin-arm64-corellium-ios: 0/2 (1 missing)

I think it's helpful to have more clarity about the builder owner. We do keep an eye on the build dashboard regularly, so it's okay if you don't do it. As long as we're able to reach the builder owner when we spot an issue, then us not having access is okay. If this happens more often, we can can revisit this.

Please let us know if you think we should change the current builder owner entry (this line) to something that's better for the future.

[eliasnaur]

Thank you for looking at this issue. I see that some of the builders are back, although it seems to be 3 out of 6 in total so far:

host-android-arm64-corellium-android: 1/1 (2 missing)
host-darwin-arm64-corellium-ios: 0/2 (1 missing)

FWIW, there are only two Android corellium hosts and two iOS corellium hosts.

I think it's helpful to have more clarity about the builder owner. We do keep an eye on the build dashboard regularly, so it's okay if you don't do it. As long as we're able to reach the builder owner when we spot an issue, then us not having access is okay. If this happens more often, we can can revisit this.

Please let us know if you think we should change the current builder owner entry (this line) to something that's better for the future.

I'd love someone on the Go team could do the resetting, which means someone should have access to the Corellium dashboard. I'm also ok doing the infrequent manual restarts.

Ideally, there should be external monitoring of Corellium and automatic restarts. Corellium is not really built for continuous integration. There's an ongoing lawsuit where Apple is trying to shut Corellium down, so motivation is low :)

[dmitshur]

FWIW, there are only two Android corellium hosts and two iOS corellium hosts.

I see. That means we can update ExpectNum accordingly.

I'd love someone on the Go team could do the resetting, which means someone should have access to the Corellium dashboard. I'm also ok doing the infrequent manual restarts.

Ideally, there should be external monitoring of Corellium and automatic restarts. Corellium is not really built for continuous integration. There's an ongoing lawsuit where Apple is trying to shut Corellium down, so motivation is low :)

Thanks for explaining.

Would you be able to send a CL adding a README https://github.com/golang/build/tree/master/env/corellium with a description of how to do the resets?

I'll close this issue since the builders are back. Thanks again.

[gopherbot]

Change https://golang.org/cl/249337 mentions this issue: dashboard: update expected number of corellium builders

[eliasnaur]

Would you be able to send a CL adding a README https://github.com/golang/build/tree/master/env/corellium with a description of how to do the resets?

Sure: https://go-review.googlesource.com/c/build/+/249358