Add Android View Hierarchy support #2440

markushi · 2022-12-22T10:23:54Z

📜 Description

Add an event processor which captures the View Hierarchy.

💡 Motivation and Context

Some refactoring was required:
Created at CurrentActivityIntegration which watches for activity lifecycle updates and updates the CurrentActivityHolder singleton - this was previously managed by the ScreenshotEventProcessor.
Now both ScreenshotEventProcessor and ViewHierarchyEventProcessor use the CurrentActivityHolder to retrieve the current activity.

Open issues:

Attachment with event.view_hierarchy type isn't shown in the Issues page relay#1704 needs fixed first before we can use the proper attachment_type

💚 How did you test it?

Unit tests

📝 Checklist

I reviewed the submitted code
I added tests to verify the changes
I updated the docs if needed
No breaking changes

🔮 Next steps

github-actions · 2022-12-22T10:25:01Z

	Messages
📖	Do not forget to update Sentry-docs with your feature once the pull request gets approved.

Generated by 🚫 dangerJS against c94745d

markushi · 2022-12-22T10:32:34Z

sentry/src/main/java/io/sentry/Attachment.java

+   * @param viewHierarchy the View Hierarchy
+   * @return the Attachment
+   */
+  public static @Nullable Attachment fromViewHierarchy(final ViewHierarchy viewHierarchy) {


@romtsn I really don't like this part

this will do the serialization on the main thread

it's accessing Sentry.getCurrentHub().getOptions().getSerializer() - which it probably shouldn't do, and which makes it hard to test

Now I'm wondering: Would it make sense to refactor the Attachment class and provide an option for having serialize-able payload?
e.g.

abstract class Attachment {} class RawAttachment extends Attachment { private final byte[] bytes; } class JsonAttachment extends Attachment { private final JsonSerializable serializable; }

re. options - I think we can just pass them as a parameter from the event processor, right? then it'd be testable.

re. serialization on main thread - I think we can offload it to another thread, but should still block and wait for it, because we're doing it only for errored events and the app is gonna crash anyway? That's what we do for screenshots as well. But I'm wondering how other SDKs did that? @marandaneto @brustolin

Imo we should block here, because the app may crash, and if we don't wait for it, then we'll lose the attachment

@markushi I do the serialization on the background, you can do the same on Android, attachments already have a callback that is only called during the event sending (and reads the bytes async).
You can check the Dart PR or the Android code.

I moved the serialization part to theViewHierarchyProcessor, looking way better now in terms of testability.
re. serialization on main thread: You're right, view hierarchies are only generated when the app crashes, thus it shouldn't be an issue. (On a another note: The Android -> Native scope sync is performing serialization on the main thread as well, maybe that's something we should fix).

For cocoa, if it's a crash we serialize it in the crash thread, which is not the main. And if is an error capture we serialize in the main thread otherwise there is a bunch of warnings that we are calling UI methods in a background thread.

github-actions · 2022-12-22T10:38:39Z

Performance metrics 🚀

	Plain	With Sentry	Diff
Startup time	372.78 ms	449.46 ms	76.68 ms
Size	1.73 MiB	2.33 MiB	619.28 KiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
`4a9c176`	320.62 ms	334.68 ms	14.06 ms
`a04f788`	321.78 ms	354.12 ms	32.35 ms
`f1150bc`	306.15 ms	306.58 ms	0.44 ms
`d00c464`	337.43 ms	387.57 ms	50.15 ms
`f55020d`	338.00 ms	370.74 ms	32.74 ms
`4a9c176`	319.77 ms	363.20 ms	43.43 ms
`703d523`	275.51 ms	323.02 ms	47.51 ms
`ecf9680`	321.55 ms	385.52 ms	63.97 ms
`b85d8aa`	289.35 ms	335.92 ms	46.56 ms
`90e9745`	314.68 ms	357.28 ms	42.60 ms

App size

Revision	Plain	With Sentry	Diff
`4a9c176`	1.73 MiB	2.33 MiB	612.69 KiB
`a04f788`	1.73 MiB	2.32 MiB	609.88 KiB
`f1150bc`	1.73 MiB	2.33 MiB	615.66 KiB
`d00c464`	1.73 MiB	2.33 MiB	613.02 KiB
`f55020d`	1.73 MiB	2.33 MiB	616.54 KiB
`4a9c176`	1.73 MiB	2.33 MiB	612.69 KiB
`703d523`	1.73 MiB	2.33 MiB	613.23 KiB
`ecf9680`	1.73 MiB	2.32 MiB	612.39 KiB
`b85d8aa`	1.73 MiB	2.32 MiB	611.62 KiB
`90e9745`	1.73 MiB	2.32 MiB	608.63 KiB

Previous results on branch: feat/view-hierarchy

Startup times

Revision	Plain	With Sentry	Diff
`f194df4`	333.84 ms	375.43 ms	41.59 ms
`8a345b2`	419.88 ms	479.98 ms	60.10 ms
`660db01`	351.49 ms	375.48 ms	23.99 ms
`8120979`	323.25 ms	370.59 ms	47.34 ms
`f96fd31`	333.66 ms	396.88 ms	63.22 ms
`8d532d2`	302.78 ms	358.23 ms	55.45 ms
`4f4ea04`	326.26 ms	373.66 ms	47.40 ms
`4a4964e`	454.85 ms	474.06 ms	19.21 ms
`2733cd6`	325.02 ms	382.14 ms	57.12 ms
`a35c0bf`	290.51 ms	367.54 ms	77.03 ms

App size

Revision	Plain	With Sentry	Diff
`f194df4`	1.73 MiB	2.33 MiB	618.12 KiB
`8a345b2`	1.73 MiB	2.33 MiB	618.02 KiB
`660db01`	1.73 MiB	2.33 MiB	618.12 KiB
`8120979`	1.73 MiB	2.33 MiB	618.10 KiB
`f96fd31`	1.73 MiB	2.33 MiB	618.12 KiB
`8d532d2`	1.73 MiB	2.33 MiB	619.13 KiB
`4f4ea04`	1.73 MiB	2.33 MiB	618.12 KiB
`4a4964e`	1.73 MiB	2.33 MiB	618.37 KiB
`2733cd6`	1.73 MiB	2.33 MiB	619.22 KiB
`a35c0bf`	1.73 MiB	2.33 MiB	619.24 KiB

codecov-commenter · 2022-12-29T09:40:24Z

Codecov Report

Base: 80.13% // Head: 80.14% // Increases project coverage by +0.00% 🎉

Coverage data is based on head (bbe99b4) compared to base (0c7bed9).
Patch coverage: 81.64% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##               main    #2440    +/-   ##
==========================================
  Coverage     80.13%   80.14%            
- Complexity     3834     3870    +36     
==========================================
  Files           310      312     +2     
  Lines         14469    14664   +195     
  Branches       1915     1939    +24     
==========================================
+ Hits          11595    11752   +157     
- Misses         2124     2153    +29     
- Partials        750      759     +9

Impacted Files	Coverage Δ
...rc/main/java/io/sentry/protocol/ViewHierarchy.java	`64.28% <64.28%> (ø)`
...ain/java/io/sentry/protocol/ViewHierarchyNode.java	`77.88% <77.88%> (ø)`
sentry/src/main/java/io/sentry/Attachment.java	`100.00% <100.00%> (ø)`
sentry/src/main/java/io/sentry/Hint.java	`93.44% <100.00%> (+0.46%)`	⬆️
sentry/src/main/java/io/sentry/JsonSerializer.java	`98.01% <100.00%> (+0.04%)`	⬆️
sentry/src/main/java/io/sentry/SentryClient.java	`86.68% <100.00%> (+0.19%)`	⬆️
...ry/src/main/java/io/sentry/SentryEnvelopeItem.java	`88.60% <100.00%> (+1.02%)`	⬆️
...java/io/sentry/internal/modules/ModulesLoader.java	`92.30% <100.00%> (ø)`

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Fixes resolve errors when performing Android Studio gradle sync

This way the CurrentActivityHolder can provide the activity to both the screenshot and viewhierarchy integration

sentry-android-core/src/main/java/io/sentry/android/core/ViewHierarchyEventProcessor.java

sentry-android-core/src/main/java/io/sentry/android/core/internal/gestures/ViewUtils.java

marandaneto · 2023-01-03T10:41:36Z

M: A few methods/params/fields are missing the nullability annotations or final when they don't change its values.

Based on PR comments

sentry/src/main/java/io/sentry/Hint.java

Co-authored-by: Roman Zavarnitsyn <rom4ek93@gmail.com>

Useful to postphone serialization of attachment payload to the point when it's needed

sentry-android-core/src/main/java/io/sentry/android/core/AndroidOptionsInitializer.java

sentry/src/main/java/io/sentry/SentryEnvelopeItem.java

romtsn

Very nice, I like how straightforward the ViewHierarchyProcessor looks 👍 🚀

sentry/src/main/java/io/sentry/Attachment.java

sentry-android-core/src/main/java/io/sentry/android/core/ViewHierarchyEventProcessor.java

sentry/src/main/java/io/sentry/SentryClient.java

krystofwoldrich · 2023-01-11T12:01:50Z

I've tried the snapshotViewHierarchy on RN. It works but I get a lot "invalid id" error logs. Any ideas about what it could cause?

The whole implementation is here getsentry/sentry-react-native@b8f7b4a (#2708).

logger.log(SentryLevel.INFO, "snapshot viewHierarchy");
final @NotNull ViewHierarchy viewHierarchy = ViewHierarchyEventProcessor.snapshotViewHierarchy(decorView);

2023-01-11 12:50:10.687 10492-10550 RNSentry                io.sentry.sample                     I  snapshot viewHierarchy
2023-01-11 12:50:10.688 10492-10550 o.sentry.sample         io.sentry.sample                     E  No package ID ff found for ID 0xffffffff.
2023-01-11 12:50:10.688 10492-10550 o.sentry.sample         io.sentry.sample                     E  No package ID ff found for ID 0xffffffff.
2023-01-11 12:50:10.688 10492-10550 o.sentry.sample         io.sentry.sample                     E  No package ID ff found for ID 0xffffffff.
2023-01-11 12:50:10.689 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x00000001.
2023-01-11 12:50:10.689 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x00000107.
2023-01-11 12:50:10.689 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x00000103.
2023-01-11 12:50:10.689 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x00000101.
2023-01-11 12:50:10.689 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x000000ff.
2023-01-11 12:50:10.689 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x00000003.
2023-01-11 12:50:10.689 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x000000f9.
2023-01-11 12:50:10.689 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x000000f7.
2023-01-11 12:50:10.689 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x000000f5.
2023-01-11 12:50:10.689 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x000000f3.
2023-01-11 12:50:10.689 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x000000d1.
2023-01-11 12:50:10.689 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x000000cf.
2023-01-11 12:50:10.690 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x00000009.
2023-01-11 12:50:10.690 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x000000cd.
2023-01-11 12:50:10.690 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x0000000f.
2023-01-11 12:50:10.690 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x00000013.
2023-01-11 12:50:10.690 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x00000019.
2023-01-11 12:50:10.690 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x00000021.
2023-01-11 12:50:10.690 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x0000001d.
2023-01-11 12:50:10.690 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x00000027.
2023-01-11 12:50:10.690 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x00000025.
2023-01-11 12:50:10.691 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x000000a3.
2023-01-11 12:50:10.691 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x0000002f.
2023-01-11 12:50:10.691 10492-10550 o.sentry.sample         io.sentry.sample                     E  Invalid ID 0x0000002d.

Gets rid of "Invalid ID 0x" logcat output spam

markushi · 2023-01-11T12:39:48Z

I've tried the snapshotViewHierarchy on RN. It works but I get a lot "invalid id" error logs. Any ideas about what it could cause?

@krystofwoldrich I think I've found the culprit which was happening when converting view IDs to their resource names (e.g. button_login), a fix is already pushed, could you give it another try?

krystofwoldrich · 2023-01-11T12:45:15Z

I've tried the snapshotViewHierarchy on RN. It works but I get a lot "invalid id" error logs. Any ideas about what it could cause?

@krystofwoldrich I think I've found the culprit which was happening when converting view IDs to their resource names (e.g. button_login), a fix is already pushed, could you give it another try?

Thanks. I will try.

Update: @markushi the No package ID ff found for ID 0xffffffff. logs are gone but the Invalid ID 0x00000001 logs are still there.

Gets rid of "Invalid ID" logcat error output

markushi · 2023-01-12T09:06:20Z

Update: @markushi the No package ID ff found for ID 0xffffffff. logs are gone but the Invalid ID 0x00000001 logs are still there.

@krystofwoldrich I just pushed another fix for that. Apparently RN creates view IDs on the fly using View.generateViewId(), and since those don't have a resource name an error was logged.

krystofwoldrich

LGTM. Works great with RN. Thanks @markushi

Add intial implemention for Android View Hierarchy support

8682a9e

markushi mentioned this pull request Dec 22, 2022

Support View hierarchy getsentry/team-mobile#64

Closed

markushi commented Dec 22, 2022

View reviewed changes

markushi changed the title ~~Add initial impl for Android View Hierarchy support~~ Add impl for Android View Hierarchy support Dec 22, 2022

Move vh attachment serialization to processor, add tests

5e82c3a

Merge branch 'main' into feat/view-hierarchy

df56bd1

markushi changed the title ~~Add impl for Android View Hierarchy support~~ Add Android View Hierarchy support Dec 29, 2022

markushi added 5 commits December 29, 2022 14:32

Add changelog

426d75a

Switch sentry-compose-helper to kotlin mutliplatform

603c073

Fixes resolve errors when performing Android Studio gradle sync

Extract Screenshot activity callbacks to CurrentActivityIntegration

ccc324b

This way the CurrentActivityHolder can provide the activity to both the screenshot and viewhierarchy integration

Fix failing compose tests

5b6d756

Merge branch 'main' into feat/view-hierarchy

152cf78

markushi marked this pull request as ready for review December 30, 2022 10:21

markushi requested review from adinauer and stefanosiano as code owners December 30, 2022 10:21

markushi added 3 commits January 3, 2023 07:47

Add missing integration tests for view hierarchy

0fd2993

Merge branch 'main' into feat/view-hierarchy

e5ab60b

Fix Changelog

95b543f

marandaneto reviewed Jan 3, 2023

View reviewed changes

sentry-android-core/src/main/java/io/sentry/android/core/ViewHierarchyEventProcessor.java Outdated Show resolved Hide resolved

marandaneto reviewed Jan 3, 2023

View reviewed changes

sentry-android-core/src/main/java/io/sentry/android/core/ViewHierarchyEventProcessor.java Outdated Show resolved Hide resolved

marandaneto reviewed Jan 3, 2023

View reviewed changes

sentry-android-core/src/main/java/io/sentry/android/core/ViewHierarchyEventProcessor.java Outdated Show resolved Hide resolved

marandaneto reviewed Jan 3, 2023

View reviewed changes

sentry-android-core/src/main/java/io/sentry/android/core/internal/gestures/ViewUtils.java Outdated Show resolved Hide resolved

romtsn mentioned this pull request Jan 3, 2023

PoC radiography integration #1998

Closed

markushi added 2 commits January 3, 2023 16:03

Adapt modifiers / UTF-8 charset handling

ed189f8

Based on PR comments

Safeguard serialization of view hierarchy

c67d132

romtsn reviewed Jan 3, 2023

View reviewed changes

sentry/src/main/java/io/sentry/Hint.java Outdated Show resolved Hide resolved

markushi and others added 5 commits January 4, 2023 07:40

Update sentry/src/main/java/io/sentry/protocol/ViewHierarchy.java

1574530

Co-authored-by: Roman Zavarnitsyn <rom4ek93@gmail.com>

Update CHANGELOG.md

b4463cf

Co-authored-by: Roman Zavarnitsyn <rom4ek93@gmail.com>

Merge branch 'main' into feat/view-hierarchy

2084f33

Improve structure based on PR comments

cbffc5c

Add bytesFactory field to attachment

9f65ca6

Useful to postphone serialization of attachment payload to the point when it's needed

romtsn reviewed Jan 9, 2023

View reviewed changes

sentry-android-core/src/main/java/io/sentry/android/core/AndroidOptionsInitializer.java Outdated Show resolved Hide resolved

romtsn reviewed Jan 9, 2023

View reviewed changes

sentry/src/main/java/io/sentry/SentryEnvelopeItem.java Outdated Show resolved Hide resolved

romtsn approved these changes Jan 9, 2023

View reviewed changes

marandaneto reviewed Jan 9, 2023

View reviewed changes

sentry/src/main/java/io/sentry/Attachment.java Outdated Show resolved Hide resolved

markushi added 4 commits January 10, 2023 11:21

Use visibility attribute on Android

040e1a0

Move ScreenshotEventProcessor init

52af78f

Set proper attachment type for view hierarchy

6077f28

Re-use attachment.getBytes() result

2c0efab

krystofwoldrich reviewed Jan 10, 2023

View reviewed changes

sentry-android-core/src/main/java/io/sentry/android/core/ViewHierarchyEventProcessor.java Outdated Show resolved Hide resolved

markushi added 2 commits January 10, 2023 16:31

Move View Hierarchy serialization from processor to SentryEnvelopeItem

7ce3c98

Merge branch 'main' into feat/view-hierarchy

e219a6f

romtsn reviewed Jan 10, 2023

View reviewed changes

sentry/src/main/java/io/sentry/SentryClient.java Show resolved Hide resolved

Remove unused field

bbe99b4

Fix don't lookup resource name when view id is not set

3d22c9f

Gets rid of "Invalid ID 0x" logcat output spam

krystofwoldrich mentioned this pull request Jan 11, 2023

Add View Hierarchy getsentry/sentry-react-native#2708

Merged

11 tasks

markushi added 2 commits January 12, 2023 09:48

Fix typo

331fb96

Fix do not look up resource names for generated view ids

c94745d

Gets rid of "Invalid ID" logcat error output

krystofwoldrich approved these changes Jan 12, 2023

View reviewed changes

markushi merged commit 9e28948 into main Jan 12, 2023

markushi deleted the feat/view-hierarchy branch January 12, 2023 10:49

gslevinkov mentioned this pull request Jan 20, 2023

NO-TICKET - Bump Sentry 6.6.0 -> 6.12.1 p2p-org/key-app-android#1141

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Android View Hierarchy support #2440

Add Android View Hierarchy support #2440

markushi commented Dec 22, 2022 •

edited

github-actions bot commented Dec 22, 2022 •

edited

markushi Dec 22, 2022 •

edited

romtsn Dec 22, 2022

marandaneto Dec 22, 2022

markushi Dec 29, 2022

brustolin Jan 4, 2023

github-actions bot commented Dec 22, 2022 •

edited

Baseline results on branch: main

Startup times

App size

Previous results on branch: feat/view-hierarchy

Startup times

App size

codecov-commenter commented Dec 29, 2022 •

edited

marandaneto commented Jan 3, 2023

romtsn left a comment

krystofwoldrich commented Jan 11, 2023 •

edited

markushi commented Jan 11, 2023 •

edited

krystofwoldrich commented Jan 11, 2023 •

edited

markushi commented Jan 12, 2023

krystofwoldrich left a comment •

edited

Add Android View Hierarchy support #2440

Add Android View Hierarchy support #2440

Conversation

markushi commented Dec 22, 2022 • edited

📜 Description

💡 Motivation and Context

💚 How did you test it?

📝 Checklist

🔮 Next steps

github-actions bot commented Dec 22, 2022 • edited

markushi Dec 22, 2022 • edited

Choose a reason for hiding this comment

romtsn Dec 22, 2022

Choose a reason for hiding this comment

marandaneto Dec 22, 2022

Choose a reason for hiding this comment

markushi Dec 29, 2022

Choose a reason for hiding this comment

brustolin Jan 4, 2023

Choose a reason for hiding this comment

github-actions bot commented Dec 22, 2022 • edited

Performance metrics 🚀

Baseline results on branch: main

Startup times

App size

Previous results on branch: feat/view-hierarchy

Startup times

App size

codecov-commenter commented Dec 29, 2022 • edited

Codecov Report

marandaneto commented Jan 3, 2023

romtsn left a comment

Choose a reason for hiding this comment

krystofwoldrich commented Jan 11, 2023 • edited

markushi commented Jan 11, 2023 • edited

krystofwoldrich commented Jan 11, 2023 • edited

markushi commented Jan 12, 2023

krystofwoldrich left a comment • edited

Choose a reason for hiding this comment

markushi commented Dec 22, 2022 •

edited

github-actions bot commented Dec 22, 2022 •

edited

markushi Dec 22, 2022 •

edited

github-actions bot commented Dec 22, 2022 •

edited

codecov-commenter commented Dec 29, 2022 •

edited

krystofwoldrich commented Jan 11, 2023 •

edited

markushi commented Jan 11, 2023 •

edited

krystofwoldrich commented Jan 11, 2023 •

edited

krystofwoldrich left a comment •

edited