Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(ios): allow larger app size diff (up to 1 MiB) #3864

Merged
merged 1 commit into from
Jun 6, 2024

Conversation

vaind
Copy link
Collaborator

@vaind vaind commented Jun 5, 2024

bumps the expected size after recent sentry-cocoa update

#skip-changelog

bumps the expected size after recent sentry-cocoa update
Copy link
Contributor

github-actions bot commented Jun 5, 2024

Android (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 382.13 ms 411.52 ms 29.39 ms
Size 17.73 MiB 19.94 MiB 2.20 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
6e8584e 447.10 ms 474.71 ms 27.61 ms
80b2ce3 385.02 ms 387.36 ms 2.34 ms
728164b 414.34 ms 449.22 ms 34.88 ms
15c80ab+dirty 336.27 ms 350.58 ms 14.31 ms
457e29f 398.10 ms 421.39 ms 23.29 ms
5bb8d5f 431.21 ms 459.40 ms 28.19 ms
d197b5c+dirty 338.94 ms 354.87 ms 15.93 ms
31fcca2 391.22 ms 414.78 ms 23.56 ms
e5c9b8b 409.02 ms 426.66 ms 17.64 ms
12427f4 393.69 ms 414.84 ms 21.14 ms

App size

Revision Plain With Sentry Diff
6e8584e 17.73 MiB 19.86 MiB 2.12 MiB
80b2ce3 17.73 MiB 19.75 MiB 2.02 MiB
728164b 17.73 MiB 19.85 MiB 2.12 MiB
15c80ab+dirty 17.73 MiB 20.04 MiB 2.31 MiB
457e29f 17.73 MiB 19.84 MiB 2.10 MiB
5bb8d5f 17.73 MiB 19.93 MiB 2.20 MiB
d197b5c+dirty 17.73 MiB 20.04 MiB 2.31 MiB
31fcca2 17.73 MiB 19.90 MiB 2.17 MiB
e5c9b8b 17.73 MiB 19.83 MiB 2.10 MiB
12427f4 17.73 MiB 19.85 MiB 2.12 MiB

Copy link
Contributor

github-actions bot commented Jun 5, 2024

Android (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 356.38 ms 379.64 ms 23.26 ms
Size 7.15 MiB 8.21 MiB 1.06 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
457e29f+dirty 591.49 ms 612.96 ms 21.47 ms
e5c9b8b+dirty 335.40 ms 360.06 ms 24.67 ms
6e8584e+dirty 383.37 ms 400.84 ms 17.47 ms
62a750b+dirty 370.78 ms 376.73 ms 5.96 ms
0db0c72+dirty 335.20 ms 351.06 ms 15.86 ms
3ffcddd+dirty 244.82 ms 295.47 ms 50.65 ms
15c80ab+dirty 276.38 ms 327.54 ms 51.17 ms
d197b5c+dirty 258.75 ms 313.61 ms 54.86 ms
5a22220+dirty 384.61 ms 419.06 ms 34.45 ms
b1e8712+dirty 322.55 ms 331.84 ms 9.29 ms

App size

Revision Plain With Sentry Diff
457e29f+dirty 7.15 MiB 8.10 MiB 981.29 KiB
e5c9b8b+dirty 7.15 MiB 8.10 MiB 980.41 KiB
6e8584e+dirty 7.15 MiB 8.13 MiB 1002.18 KiB
62a750b+dirty 7.15 MiB 8.21 MiB 1.06 MiB
0db0c72+dirty 7.15 MiB 8.04 MiB 911.02 KiB
3ffcddd+dirty 7.15 MiB 8.04 MiB 912.17 KiB
15c80ab+dirty 7.15 MiB 8.09 MiB 966.13 KiB
d197b5c+dirty 7.15 MiB 8.09 MiB 962.72 KiB
5a22220+dirty 7.15 MiB 8.21 MiB 1.06 MiB
b1e8712+dirty 7.15 MiB 8.04 MiB 912.27 KiB

Copy link
Contributor

github-actions bot commented Jun 5, 2024

iOS (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1229.55 ms 1229.83 ms 0.28 ms
Size 2.36 MiB 3.03 MiB 678.67 KiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
6e8584e+dirty 1274.50 ms 1296.82 ms 22.32 ms
d361d38+dirty 1246.04 ms 1267.12 ms 21.08 ms
2534337+dirty 1225.08 ms 1230.26 ms 5.17 ms
5a22220+dirty 1209.49 ms 1220.94 ms 11.45 ms
5bb8d5f+dirty 1235.47 ms 1237.39 ms 1.92 ms
76d1baf+dirty 1244.10 ms 1268.52 ms 24.42 ms
0db0c72+dirty 1275.02 ms 1285.84 ms 10.82 ms
3ffcddd+dirty 1244.47 ms 1264.14 ms 19.67 ms
52a8031+dirty 1280.88 ms 1289.78 ms 8.90 ms
86d6d2c+dirty 1267.55 ms 1286.21 ms 18.66 ms

App size

Revision Plain With Sentry Diff
6e8584e+dirty 2.36 MiB 2.88 MiB 533.17 KiB
d361d38+dirty 2.36 MiB 2.85 MiB 499.84 KiB
2534337+dirty 2.36 MiB 2.88 MiB 525.47 KiB
5a22220+dirty 2.36 MiB 2.92 MiB 570.21 KiB
5bb8d5f+dirty 2.36 MiB 2.92 MiB 570.22 KiB
76d1baf+dirty 2.36 MiB 2.82 MiB 469.45 KiB
0db0c72+dirty 2.36 MiB 2.84 MiB 487.01 KiB
3ffcddd+dirty 2.36 MiB 2.84 MiB 489.60 KiB
52a8031+dirty 2.36 MiB 2.82 MiB 469.44 KiB
86d6d2c+dirty 2.36 MiB 2.82 MiB 462.82 KiB

@krystofwoldrich
Copy link
Member

@vaind Thank you this is a good catch.

A lot of the times these tests fail on the start up time of the app without Sentry is higher than with Sentry. And so it's easy to overlook that something else was broken.

@krystofwoldrich krystofwoldrich changed the title ci: Update metrics-ios.yml ci(ios): Allow larger iOS app size diff (up to 1 MiB) Jun 6, 2024
@krystofwoldrich krystofwoldrich changed the title ci(ios): Allow larger iOS app size diff (up to 1 MiB) ci(ios): allow larger iOS app size diff (up to 1 MiB) Jun 6, 2024
@krystofwoldrich krystofwoldrich changed the title ci(ios): allow larger iOS app size diff (up to 1 MiB) ci(ios): allow larger app size diff (up to 1 MiB) Jun 6, 2024
@krystofwoldrich krystofwoldrich merged commit 6a87ede into main Jun 6, 2024
63 of 66 checks passed
@krystofwoldrich krystofwoldrich deleted the vaind-patch-1 branch June 6, 2024 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants