Use the Pixel 7/API 33 device in the Firebase test lab script

[jason-simmons]

This test is currently failing consistently when running with Pixel 8/API 34

[flutter-dashboard]

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption to this rule, contact "@test-exemption-reviewer" in the #hackers channel in Chat (don't just cc them here, they won't see it! Use Discord!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

[gmackall]

LGTM to get the tree green, though I would be curious if "husky" (pixel 8 pro) passes (ideally we wouldn't regress off of api34)

[jason-simmons]

Tried a run with model=husky,version=34. It also resulted in an infrastructure error on FTL:
https://console.firebase.google.com/project/flutter-infra-staging/testlab/histories/bh.6b85e5d132218e8f/matrices/8478154452546318566

[gmackall]

I was going to comment that it is interesting that they are still working on the flutter/flutter dashboard
https://flutter-dashboard.appspot.com/#/build
But they are just failing? But not blocking the tree...
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20firebase_android_embedding_v2_smoke_test/17576/overview

┌──────────────┬────────────────────────┬────────────────────────┐
│   OUTCOME    │    TEST_AXIS_VALUE     │      TEST_DETAILS      │
├──────────────┼────────────────────────┼────────────────────────┤
│ Passed       │ griffin-24-en-portrait │ --                     │
│ Passed       │ redfin-30-en-portrait  │ --                     │
│ Inconclusive │ shiba-34-en-portrait   │ Infrastructure failure │
└──────────────┴────────────────────────┴────────────────────────┘

[reidbaker]

@jason-simmons does this indicate a real bug on api 34 or something that is a test only failure?

[reidbaker]

Unless there is evidence that this is a test only failure I dont think changing what we test to get the tree green is the right decision.

[zanderso]

I filed flutter/flutter#143590 for the framework CI non-failure.

[zanderso]

@jason-simmons does this indicate a real bug on api 34 or something that is a test only failure?

The only thing we know at this time is that Firebase test lab is reporting an infrastructure failure for that configuration.

[zanderso]

To be clear though, I don't know what you mean by a "test only failure".

[jason-simmons]

Firebase Test Lab is reporting the test status as "inconclusive", which IIUC typically means an infrastructure issue within FTL.

I did not notice anything in the FTL logs that explained the reason for the failure. Apps built with the ToT engine seem to be behaving as expected on my local Pixel 6a running API 34.

[reidbaker]

To be clear though, I don't know what you mean by a "test only failure".

I was trying to distinguish between and issue that was present in our code and an issue that was present in our testing code.

Gray and I paired on 2 prs to try to help prove that this is an FTL hardware issue
flutter/flutter#143596 and flutter/flutter#143595
Both show that this is likely something on their end.

Additionally flutter/flutter#143590 (comment) tells us that the engine and framework respond differently to FTL reporting infra issues. There is a error code we can use to ignore failures if we want.