daily job running mobile example on real devices

[mockersf]

Objective

• Test mobile example on real devices

Solution

• Use BrowserStack to have access to real devices
  • App Automate to run the example
  • App Percy to compare the screenshot
• Added a daily/manual CI job that will build for iOS and Android, send the apps to BrowserStack, run the app on one iOS device and one Android device, capture a screenshot, send it for visual validation, and archive it in the GitHub action

Example run: https://github.com/mockersf/bevy/actions/runs/4521883534

They currently have a bug with the settings to view snapshots, they should be public. I'll raise it to them, and if they don't fix it in time it's possible to work around for everyone to view the results through their API.

@cart to get this to work, you'll need

• to set up an account on BrowserStack
• add the secrets BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY to the Bevy repo
• create a project in Percy
• add the secret PERCY_TOKEN to the Bevy repo and modify the project name line 122 in the Daily.yml file

[mockersf]

CODE_SIGN_IDENTITY = "" is needed to be able to build the app without signing it in CI

[alice-i-cecile]

Can you leave a comment in this type of file?

[mockersf]

The file is not meant to be read, it's generated and managed through xcode. It's also a setting specific to Bevy as an engine, someone creating a game and targeting iOS will have a signing identity. As Bevy is not paying the 100$/year apple development subscription, we don't have one.

[mockersf]

this is so that both Android and iOS have the same package name

[alice-i-cecile]

Controversial label added because of the third party integration. Strongly in favor of this though: there's a been quite a few surprising regressions, and if we're going to get mobile support working in earnest we need to be testing on real devices.

[cart]

(@mockersf sent me the secrets. I added them to the Bevy Org and gave the bevy repo access)