Update benchmark cpu approach #2473
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added sustained performance mode on profiling benchmarks, but not on sdk init changed cpu time reading from Process.getElapsedCpuTime() to reading /proc/self/stat file
reverted threshold from 5.5% to 5%
stefanosiano
requested review from
adinauer,
markushi and
romtsn
as code owners
Contributor
Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 2b4a65c | 359.07 ms | 399.76 ms | 40.69 ms |
| faa106a | 427.66 ms | 432.25 ms | 4.59 ms |
| b5825ba | 332.64 ms | 390.43 ms | 57.79 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 2b4a65c | 1.73 MiB | 2.33 MiB | 619.47 KiB |
| faa106a | 1.73 MiB | 2.33 MiB | 619.47 KiB |
| b5825ba | 1.73 MiB | 2.33 MiB | 619.46 KiB |
Codecov ReportBase: 80.14% // Head: 80.14% // No change to project coverage 👍
Additional details and impacted files@@ Coverage Diff @@
## main #2473 +/- ##
=========================================
Coverage 80.14% 80.14%
Complexity 3872 3872
=========================================
Files 312 312
Lines 14669 14669
Branches 1941 1941
=========================================
Hits 11756 11756
Misses 2153 2153
Partials 760 760 Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
romtsn
reviewed
Jan 16, 2023
...rk/src/androidTest/java/io/sentry/uitest/android/benchmark/util/BenchmarkComparisonResult.kt
Outdated
Show resolved
Hide resolved
romtsn
approved these changes
Jan 16, 2023
Member
romtsn
left a comment
romtsn
left a comment
There was a problem hiding this comment.
Great stuff, especially the sustainedPerformanceMode flag 🚀
…rk/src/androidTest/java/io/sentry/uitest/android/benchmark/util/BenchmarkComparisonResult.kt Co-authored-by: Roman Zavarnitsyn <rom4ek93@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
📜 Description
Cpu usage in benchmark has now been converted by reading the /proc/self/stat/ file.
Updated devices used in Saucelabs for benchmarking.
Decreased acceptance threshold from 5.5% to 5%
#skip-changelog
💡 Motivation and Context
Previous approach was reading global cpu time to make comparisons.
New approach reads /proc/self/stat/ file, which returns the cpu time spent by the application only, giving more accurate results.
Also, the
sustainedPerformanceModehas been enabled in the benchmarks, which prevents thermal throttling, increasing accuracy even more.💚 How did you test it?
📝 Checklist
🔮 Next steps