Skip to content

Update robo test devices MAPSAND-268#1472

Merged
pengdev merged 9 commits intomainfrom
peng-update-robotest-device
Jul 1, 2022
Merged

Update robo test devices MAPSAND-268#1472
pengdev merged 9 commits intomainfrom
peng-update-robotest-device

Conversation

@pengdev
Copy link
Copy Markdown
Member

@pengdev pengdev commented Jun 30, 2022

Summary of changes

Update to use the following robo test devices to run robo tests:

  • 1 old API (the lower the better)
  • 1 new API (the higher the better)
  • 1 x86-64 emulator(Pixel2)
  • API level 23 which is known to work weird

User impact (optional)

Pull request checklist:

  • Briefly describe the changes in this PR.
  • Include before/after visuals or gifs if this PR includes visual changes.
  • Write tests for all new functionality. If tests were not written, please explain why.
  • Optimize code for java consumption (@JvmOverloads, @file:JvmName, etc).
  • Add example if relevant.
  • Document any changes to public APIs.
  • Run make update-api to update generated api files, if there's public API changes, otherwise the verify-api-* CI steps might fail.
  • Update CHANGELOG.md or use the label 'skip changelog', otherwise check changelog CI step will fail.
  • If this PR is a v10.[version] release branch fix / enhancement, merge it to main firstly and then port to v10.[version] release branch.

Fixes: < Link to related issues that will be fixed by this pull request, if they exist >

PRs must be submitted under the terms of our Contributor License Agreement CLA.

@pengdev pengdev self-assigned this Jun 30, 2022
@pengdev pengdev changed the title Change robo test device from virtual Pixel2 to Pixel3. Update robo test devices. Jun 30, 2022
@ank27
Copy link
Copy Markdown
Contributor

ank27 commented Jun 30, 2022

Can we also add command to fetch model list so we know availability of devices. i.e. add gcloud firebase test android models list , before running gcloud firebase test android run ... ?

@pengdev pengdev marked this pull request as ready for review June 30, 2022 11:22
@pengdev pengdev requested a review from a team as a code owner June 30, 2022 11:22
Copy link
Copy Markdown
Contributor

@kiryldz kiryldz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just an idea - perhaps simply leave following devices to save costs (and also assuming that robo tests did not catch anything useful for quite a time already):

  • api 23 - differs from APIs above in terms for at least Android animators
  • pixel 3, api 30 - assuming it's x86 to validate no issues there
  • api 31, pretty fresh one

Other 2 devices are a bit questionable for me but in general LGTM to leave them as well

Comment thread .circleci/config.yml Outdated
@pengdev pengdev changed the title Update robo test devices. Update robo test devices MAPSAND-268 Jul 1, 2022
@pengdev
Copy link
Copy Markdown
Member Author

pengdev commented Jul 1, 2022

The reason behind the virtual device failure is that there’s an ongoing incident on Firebase test lab:

Firebase Test Lab is experiencing inclusive Robo test results running on virtual API 30 and 31. We currently estimate that 100% of users of Robo tests on virtual API 30 or 31 are affected. We are currently investigating the cause.

Firebase Status Dashboard

I've changed the virtual device to Pixel 2 at API level 29 to avoid the issue.

@pengdev pengdev merged commit 149e330 into main Jul 1, 2022
@pengdev pengdev deleted the peng-update-robotest-device branch July 1, 2022 11:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants