Skip to content

Call setSustainedPerformanceMode for better benchmarks#2329

Merged
stefanosiano merged 1 commit intomainfrom
tests/benchmark-sustained-mode
Nov 4, 2022
Merged

Call setSustainedPerformanceMode for better benchmarks#2329
stefanosiano merged 1 commit intomainfrom
tests/benchmark-sustained-mode

Conversation

@stefanosiano
Copy link
Copy Markdown
Contributor

📜 Description

Added window.setSustainedPerformanceMode(true) on benchmark activity, to get better results.
Reduced espresso default timeout to 10 seconds
Updated list of devices for benchmarks

#skip-changelog

💡 Motivation and Context

setSustainedPerformanceMode is one of the things the android benchmark runner (which we cannot use due to problems on Saucelabs) does. It tries to limit the thermal throttling by reducing the overall performance. https://developer.android.com/reference/kotlin/androidx/benchmark/junit4/AndroidBenchmarkRunner#clock-stability
This will result in more predictable benchmarks, as currently there are few runs failing.
Saucelabs devices for benchmarks were updated: the Google Pixel 3 was replaced because consistently timing out

💚 How did you test it?

📝 Checklist

  • I reviewed the submitted code
  • I added tests to verify the changes
  • I updated the docs if needed
  • No breaking changes

🔮 Next steps

… to get better results

reduced espresso default timeout to 10 seconds
updated list of devices for benchmarks
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Nov 3, 2022

Performance metrics 🚀

  Plain With Sentry Diff
Startup time 369.58 ms 405.46 ms 35.88 ms
Size 1.73 MiB 2.32 MiB 608.53 KiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
f809aac 301.51 ms 346.60 ms 45.09 ms
90e9745 314.68 ms 357.28 ms 42.60 ms

App size

Revision Plain With Sentry Diff
f809aac 1.73 MiB 2.32 MiB 608.63 KiB
90e9745 1.73 MiB 2.32 MiB 608.63 KiB

@stefanosiano stefanosiano marked this pull request as ready for review November 3, 2022 17:38
Copy link
Copy Markdown
Member

@markushi markushi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@stefanosiano stefanosiano merged commit 34811f4 into main Nov 4, 2022
@stefanosiano stefanosiano deleted the tests/benchmark-sustained-mode branch November 4, 2022 06:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants