Two benchmarks that specify the same device in .ci.yaml are running on different devices.

[zanderso]

In the.ci.yaml file Linux_samsung_s10 new_gallery__transition_perf and Linux_samsung_s10 new_gallery_impeller__transition_perf both specify tags ["devicelab", "android", "linux", "samsung", "s10"], which I would have guessed would mean that both would be running on Samsung S10 devices.

However, looking at both SkiaPerf and at the pages for the builders, they are running on different device types.

For example here, the impeller version ran on flutter-devicelab-linux-18, which has device type SM-A025V.

But here, the non-Impeller version ran flutter-devicelab-linux-20, which has device type SM-G973U1.

We are going to be able to need to compare Impeller to non-Impeller across a range of benchmarks and devices, so I'd really like to be 100% sure about the devices that we're running on. How can I do that?

@keyonghan

[keyonghan]

We have been relying on device_os dimension for task scheduling, but it turns out Samsung A02 and Samsung S10 may share the same device_os (ro.build.id):

testbed     device_os           device_type
linux-18    RP1A.200720.012     SM-A025V
linux-20    RP1A.200720.012     SM-G973U1
linux-26    SP1A.210812.016     SM-G973U1

But device_type is consistent to distinguish A02 from S10.

@zanderso Does it make sense to switch to device_type or do we not care the device_os?

[zanderso]

For the purposes of benchmarking, the most important distinguishing feature is the device type.

[keyonghan]

#105322 to fix.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.