Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build/dashboard: darwin-arm-mg912baios and darwin-arm64-mn4m2zdaios builders consistently failing #31497

Open
bcmills opened this issue Apr 16, 2019 · 15 comments

Comments

@bcmills
Copy link
Member

commented Apr 16, 2019

Starting around https://build.golang.org/log/776c688e90aaa17f09782242647391a368eeabc7 (Apr. 9), the two darwin-arm* builders have been failing with lots of messages of the form

go_darwin_arm_exec: GOIOS_DEV_ID not set
run $GOROOT/misc/ios/detect.go to attempt to autodetect
FAIL	archive/tar	0.017s

It seems like that may be due to a change in the builder configuration, but I'm not really sure.

@eliasnaur, could you look into it (or loop in someone who can)?

@eliasnaur

This comment has been minimized.

Copy link
Contributor

commented Apr 16, 2019

Yes, I've pinged @steeve. I believe he's on it.

@steeve

This comment has been minimized.

Copy link
Contributor

commented Apr 19, 2019

I am. I'll try to completely factory reset the arm64 phone. arm32 should be ok I think

@bradfitz

This comment has been minimized.

Copy link
Member

commented Apr 24, 2019

Any update?

@steeve

This comment has been minimized.

Copy link
Contributor

commented Apr 24, 2019

Yes, the builders seems ok now.
I did a full reset and update to iOS 12.2 on the arm64 phone.

@bcmills

This comment has been minimized.

Copy link
Member Author

commented Oct 11, 2019

Looks like darwin-arm64-mn4m2zdaios is almost back up, but it's failing in the std and cmd tests due to #34847 and two very terse failures for runtime and runtime/pprof (perhaps OOM errors?): https://build.golang.org/log/048cb165b4cd25eb76cb1c4c9b6bb2b2b93a47ce

@bcmills

This comment has been minimized.

Copy link
Member Author

commented Oct 11, 2019

darwin-arm-mg912baios seems to be unplugged (https://build.golang.org/log/532482aa5995eaeb33bbd8dbeca8064b22a11014):

No device found with udid 8e5c23a5d0843d1ffe164ea0b2f2500599c3ebff, is it plugged in?
@steeve

This comment has been minimized.

Copy link
Contributor

commented Oct 11, 2019

@bcmills yes, the 32 bits device is having issues, I'm trying to get another device to test.

@steeve

This comment has been minimized.

Copy link
Contributor

commented Oct 11, 2019

I bisected the problem in ideviceinstaller. The problem is libimobiledevice/ideviceinstaller@4a7f0b0

I have opened libimobiledevice/ideviceinstaller#119 to track it. In the mean time I'm going to revert that commit on the build machine.

@steeve

This comment has been minimized.

Copy link
Contributor

commented Oct 11, 2019

I have downgraded ideviceinstaller, and the builder now seems to run ok. I'll monitor.

@steeve

This comment has been minimized.

Copy link
Contributor

commented Oct 11, 2019

The builder is back online for the two archs, and now they are failing for good reasons

@bcmills

This comment has been minimized.

Copy link
Member Author

commented Oct 15, 2019

now they are failing for good reasons

It looks like the current failures are not sending output back to the runner. Is there some change that needs to be made to extract it from the device on failure?

Are there issues filed for the current failures?

@steeve

This comment has been minimized.

Copy link
Contributor

commented Oct 15, 2019

I am not aware regarding issues for the current failures? Probably because of lack of output.
Not sure how to extract it though, @eliasnaur?

@eliasnaur

This comment has been minimized.

Copy link
Contributor

commented Oct 15, 2019

I don't know of any way than manually running the tests. See

#34751 (comment)
#34751 (comment)

@steeve

This comment has been minimized.

Copy link
Contributor

commented Oct 15, 2019

Thanks, I'll do it tomorrow!

@eliasnaur

This comment has been minimized.

Copy link
Contributor

commented Oct 16, 2019

FWIW, you might find #34501 (comment) useful. You can now use TRY= to target a particular set of builders.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.