misc(ci): Update e2e tests driver #3499

krystofwoldrich · 2024-01-08T09:19:14Z

📢 Type of change

Bugfix
New feature
Enhancement
Refactoring

📜 Description

The Appium v2 stable version is out, our e2e tests were still using the beta. I could not run the beta on my Mac anymore.

💡 Motivation and Context

💚 How did you test it?

run test locally, ci

📝 Checklist

I reviewed submitted code
I added tests to verify changes
No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled
All tests passing
No breaking changes

🔮 Next steps

#skip-changelog

This reverts commit ad46638.

Co-authored-by: LucasZF <lucas-zimerman1@hotmail.com>

github-actions · 2024-01-08T09:36:22Z

Android (legacy) Performance metrics 🚀

	Plain	With Sentry	Diff
Startup time	398.67 ms	428.74 ms	30.06 ms
Size	17.73 MiB	19.93 MiB	2.19 MiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
`abb7058`	370.27 ms	389.58 ms	19.31 ms
`3ffcddd`	302.92 ms	315.80 ms	12.88 ms
`8900e1a`+dirty	430.68 ms	456.13 ms	25.44 ms
`d361d38`	354.10 ms	381.69 ms	27.59 ms
`9a3ca65`+dirty	326.93 ms	330.14 ms	3.21 ms
`b1e8712`	462.11 ms	465.71 ms	3.60 ms
`2534337`	394.15 ms	415.12 ms	20.97 ms
`34aba08`	328.10 ms	342.84 ms	14.74 ms
`76d1baf`+dirty	335.72 ms	355.52 ms	19.80 ms
`31fcca2`	391.22 ms	414.78 ms	23.56 ms

App size

Revision	Plain	With Sentry	Diff
`abb7058`	17.73 MiB	19.83 MiB	2.10 MiB
`3ffcddd`	17.73 MiB	19.75 MiB	2.02 MiB
`8900e1a`+dirty	17.73 MiB	19.75 MiB	2.01 MiB
`d361d38`	17.73 MiB	19.81 MiB	2.08 MiB
`9a3ca65`+dirty	17.73 MiB	20.04 MiB	2.31 MiB
`b1e8712`	17.73 MiB	19.75 MiB	2.02 MiB
`2534337`	17.73 MiB	19.84 MiB	2.11 MiB
`34aba08`	17.73 MiB	19.80 MiB	2.07 MiB
`76d1baf`+dirty	17.73 MiB	20.04 MiB	2.31 MiB
`31fcca2`	17.73 MiB	19.90 MiB	2.17 MiB

krystofwoldrich · 2024-01-08T09:56:16Z

0.73 e2e tests build are failing due to Pod install error verification checksum was incorrect at boost (1.83.0) facebook/react-native#42109 (comment)

github-actions · 2024-01-09T10:59:59Z

Android (new) Performance metrics 🚀

	Plain	With Sentry	Diff
Startup time	360.30 ms	409.50 ms	49.20 ms
Size	7.15 MiB	8.11 MiB	988.68 KiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
`d0bf494`+dirty	253.73 ms	308.23 ms	54.49 ms
`3853f43`+dirty	278.12 ms	338.72 ms	60.60 ms
`e73f4ed`+dirty	262.98 ms	311.02 ms	48.04 ms
`9c48b2c`+dirty	270.82 ms	321.12 ms	50.30 ms
`27ef4ee`+dirty	296.71 ms	351.00 ms	54.29 ms
`6e8584e`+dirty	383.37 ms	400.84 ms	17.47 ms
`acadc0f`+dirty	259.04 ms	304.67 ms	45.63 ms
`8900e1a`+dirty	371.40 ms	377.70 ms	6.31 ms
`e2b64fe`+dirty	258.82 ms	304.26 ms	45.44 ms
`e5c9b8b`+dirty	335.40 ms	360.06 ms	24.67 ms

App size

Revision	Plain	With Sentry	Diff
`d0bf494`+dirty	7.15 MiB	8.04 MiB	910.85 KiB
`3853f43`+dirty	7.15 MiB	8.08 MiB	959.34 KiB
`e73f4ed`+dirty	7.15 MiB	8.09 MiB	965.94 KiB
`9c48b2c`+dirty	7.15 MiB	8.07 MiB	947.16 KiB
`27ef4ee`+dirty	7.15 MiB	8.08 MiB	959.49 KiB
`6e8584e`+dirty	7.15 MiB	8.13 MiB	1002.18 KiB
`acadc0f`+dirty	7.15 MiB	8.03 MiB	903.20 KiB
`8900e1a`+dirty	7.15 MiB	8.03 MiB	901.79 KiB
`e2b64fe`+dirty	7.15 MiB	8.07 MiB	947.16 KiB
`e5c9b8b`+dirty	7.15 MiB	8.10 MiB	980.41 KiB

github-actions · 2024-01-09T11:06:19Z

iOS (new) Performance metrics 🚀

	Plain	With Sentry	Diff
Startup time	1226.59 ms	1234.48 ms	7.89 ms
Size	2.92 MiB	3.48 MiB	575.52 KiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
`80b2ce3`+dirty	1245.12 ms	1262.04 ms	16.92 ms
`d7401ac`+dirty	1288.10 ms	1289.54 ms	1.44 ms
`e5c9b8b`+dirty	1276.90 ms	1280.92 ms	4.02 ms
`acadc0f`+dirty	1271.12 ms	1272.28 ms	1.16 ms
`728164b`+dirty	1280.06 ms	1285.26 ms	5.20 ms
`3ffcddd`+dirty	1272.22 ms	1273.98 ms	1.76 ms
`9a3ca65`+dirty	1276.40 ms	1279.14 ms	2.74 ms
`457e29f`+dirty	1256.71 ms	1258.50 ms	1.79 ms
`12427f4`+dirty	1224.90 ms	1231.40 ms	6.50 ms
`2534337`+dirty	1220.87 ms	1221.47 ms	0.60 ms

App size

Revision	Plain	With Sentry	Diff
`80b2ce3`+dirty	2.92 MiB	3.40 MiB	492.75 KiB
`d7401ac`+dirty	2.92 MiB	3.40 MiB	488.06 KiB
`e5c9b8b`+dirty	2.92 MiB	3.43 MiB	524.50 KiB
`acadc0f`+dirty	2.92 MiB	3.39 MiB	487.34 KiB
`728164b`+dirty	2.92 MiB	3.44 MiB	533.26 KiB
`3ffcddd`+dirty	2.92 MiB	3.40 MiB	494.39 KiB
`9a3ca65`+dirty	2.92 MiB	3.37 MiB	464.32 KiB
`457e29f`+dirty	2.92 MiB	3.43 MiB	524.75 KiB
`12427f4`+dirty	2.92 MiB	3.44 MiB	533.29 KiB
`2534337`+dirty	2.92 MiB	3.43 MiB	529.76 KiB

github-actions · 2024-01-09T14:47:04Z

iOS (legacy) Performance metrics 🚀

	Plain	With Sentry	Diff
Startup time	1227.19 ms	1233.45 ms	6.26 ms
Size	2.36 MiB	2.92 MiB	569.94 KiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
`80b2ce3`+dirty	1265.92 ms	1268.60 ms	2.69 ms
`d7401ac`+dirty	1252.38 ms	1275.04 ms	22.66 ms
`e5c9b8b`+dirty	1258.57 ms	1267.32 ms	8.75 ms
`acadc0f`+dirty	1264.38 ms	1290.06 ms	25.68 ms
`728164b`+dirty	1256.10 ms	1259.08 ms	2.98 ms
`3ffcddd`+dirty	1244.47 ms	1264.14 ms	19.67 ms
`9a3ca65`+dirty	1247.06 ms	1274.58 ms	27.52 ms
`457e29f`+dirty	1253.94 ms	1269.18 ms	15.24 ms
`12427f4`+dirty	1267.15 ms	1271.30 ms	4.15 ms
`2534337`+dirty	1225.08 ms	1230.26 ms	5.17 ms

App size

Revision	Plain	With Sentry	Diff
`80b2ce3`+dirty	2.36 MiB	2.84 MiB	486.98 KiB
`d7401ac`+dirty	2.36 MiB	2.83 MiB	481.14 KiB
`e5c9b8b`+dirty	2.36 MiB	2.87 MiB	520.43 KiB
`acadc0f`+dirty	2.36 MiB	2.83 MiB	480.37 KiB
`728164b`+dirty	2.36 MiB	2.88 MiB	530.38 KiB
`3ffcddd`+dirty	2.36 MiB	2.84 MiB	489.60 KiB
`9a3ca65`+dirty	2.36 MiB	2.82 MiB	462.89 KiB
`457e29f`+dirty	2.36 MiB	2.87 MiB	520.67 KiB
`12427f4`+dirty	2.36 MiB	2.88 MiB	530.38 KiB
`2534337`+dirty	2.36 MiB	2.88 MiB	525.47 KiB

krystofwoldrich · 2024-03-20T14:15:56Z

test/e2e/test/utils/fetchEvent.ts

@@ -1,5 +1,4 @@
 import type { Event } from '@sentry/types';
-import fetch, { Request } from 'node-fetch';


Fetch is in Node.JS since v18

lucas-zimerman

LGTM!

krystofwoldrich and others added 30 commits October 16, 2023 10:16

Bump e2e tests to 0.73.0-rc.2

acbf6b2

Upgrade java version for the new react native

953aba0

Include prerelease xcode patch script

28b0acb

Merge remote-tracking branch 'origin/main' into kw-rn-0-73

8df10c8

Update default xcode version to 14.3

ad46638

Revert "Update default xcode version to 14.3"

efd498d

This reverts commit ad46638.

Bump ro rc 3

1e876e3

Merge branch 'main' into kw-rn-0-73

270edc4

Upgrade android to 0.73 (not all libs are compatible yet)

73a3774

Merge branch 'main' into kw-rn-0-73

899523c

Upgrade to latest rc, add safe area patch, fix ios project

172341b

Remove start package build phase

56fb902

fix android sample

70b05a5

revert sdk change

daaf8af

bump ci to node 18

c04c7c7

bump java to v17

03f4dca

Add x86_64 to bundle platforms

6198f81

keep java 11 for older builds

f31943b

Fix min ios version RN 0.73 cocoa tester

b9c4968

fix codegen java version

a6c82da

fix e2e build test 0.65 node version

4a27dcd

Keep node 18 for SDK build, bump down for app build

961a9b0

Remove hermes from cocoa tester as no engine is needed

c1626f0

Merge branch 'main' into kw-rn-0-73

31293a0

bump to 0.73.0 stable release

9d76205

Apply suggestions from code review

98f912a

Co-authored-by: LucasZF <lucas-zimerman1@hotmail.com>

chore(internal): Use new DSN after leaked token

9bcd0dd

Merge branch 'kw-new-dsn' into kw-rn-0-73

34f148f

Merge remote-tracking branch 'origin/main' into kw-rn-0-73

0e78db6

chore(internal): Update e2e tests driver

36f25b0

krystofwoldrich self-assigned this Jan 8, 2024

Base automatically changed from kw-rn-0-73 to main January 8, 2024 10:12

krystofwoldrich added 2 commits January 9, 2024 10:45

chore(rn): Update repository to RN 0.73.2

80d5b84

Merge branch 'kw-update-rn-0-73-2' into kw-bump-e2e-tests-deps

b46ebf8

krystofwoldrich added 5 commits March 18, 2024 18:12

Merge branch 'main' into kw-bump-e2e-tests-deps

1eef7e1

fix fetch

2efa751

bump e2e tests runner to node 20 to fix jest SIGSEGV

df83029

fix lint

37b7171

fix yarn lock

b8bd80f

krystofwoldrich marked this pull request as ready for review March 18, 2024 22:14

krystofwoldrich requested a review from lucas-zimerman as a code owner March 18, 2024 22:14

krystofwoldrich changed the title ~~chore(internal): Update e2e tests driver~~ misc(ci): Update e2e tests driver Mar 18, 2024

krystofwoldrich commented Mar 20, 2024

View reviewed changes

stefanosiano approved these changes Mar 20, 2024

View reviewed changes

lucas-zimerman approved these changes Mar 20, 2024

View reviewed changes

lucas-zimerman merged commit 93ab173 into main Mar 20, 2024
56 of 58 checks passed

lucas-zimerman deleted the kw-bump-e2e-tests-deps branch March 20, 2024 14:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

misc(ci): Update e2e tests driver #3499

misc(ci): Update e2e tests driver #3499

krystofwoldrich commented Jan 8, 2024

github-actions bot commented Jan 8, 2024 •

edited

Loading

Baseline results on branch: main

Startup times

App size

krystofwoldrich commented Jan 8, 2024

github-actions bot commented Jan 9, 2024

Baseline results on branch: main

Startup times

App size

github-actions bot commented Jan 9, 2024 •

edited

Loading

Baseline results on branch: main

Startup times

App size

github-actions bot commented Jan 9, 2024 •

edited

Loading

Baseline results on branch: main

Startup times

App size

krystofwoldrich Mar 20, 2024

lucas-zimerman left a comment

		@@ -1,5 +1,4 @@
		import type { Event } from '@sentry/types';
		import fetch, { Request } from 'node-fetch';

misc(ci): Update e2e tests driver #3499

misc(ci): Update e2e tests driver #3499

Conversation

krystofwoldrich commented Jan 8, 2024

📢 Type of change

📜 Description

💡 Motivation and Context

💚 How did you test it?

📝 Checklist

🔮 Next steps

github-actions bot commented Jan 8, 2024 • edited Loading

Android (legacy) Performance metrics 🚀

Baseline results on branch: main

Startup times

App size

krystofwoldrich commented Jan 8, 2024

github-actions bot commented Jan 9, 2024

Android (new) Performance metrics 🚀

Baseline results on branch: main

Startup times

App size

github-actions bot commented Jan 9, 2024 • edited Loading

iOS (new) Performance metrics 🚀

Baseline results on branch: main

Startup times

App size

github-actions bot commented Jan 9, 2024 • edited Loading

iOS (legacy) Performance metrics 🚀

Baseline results on branch: main

Startup times

App size

krystofwoldrich Mar 20, 2024

Choose a reason for hiding this comment

lucas-zimerman left a comment

Choose a reason for hiding this comment

github-actions bot commented Jan 8, 2024 •

edited

Loading

github-actions bot commented Jan 9, 2024 •

edited

Loading

github-actions bot commented Jan 9, 2024 •

edited

Loading