fix: add an empty AccessibilityNodeInfo creation for future androidx.test.uiautomator:uiautomator:2.3.0 update #589

KazuCocoa · 2023-12-31T08:52:20Z

Investigating appium/appium#19597

According to the androidx.test.uiautomator:uiautomator:2.3.0 beta01, UiObject2's constructor has changed as below, thus we must pass an empty AccessibilityNodeInfo instance as node instead of null.

    private UiObject2(UiDevice device, BySelector selector, AccessibilityNodeInfo cachedNode) {
        mDevice = device;
        mSelector = selector;
        mCachedNode = cachedNode;
        mGestureController = GestureController.getInstance(device);

        // Fetch and cache display information. This is safe as moving the underlying view to
        // another display would invalidate the cached node and require recreating this UiObject2.
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) {
            AccessibilityWindowInfo window = Api21Impl.getWindow(cachedNode);
            mDisplayId = window == null ? Display.DEFAULT_DISPLAY : Api30Impl.getDisplayId(window);
        } else {
            mDisplayId = Display.DEFAULT_DISPLAY;
        }
        Context uiContext = device.getUiContext(mDisplayId);
        int densityDpi = uiContext.getResources().getConfiguration().densityDpi;
        mDisplayDensity = (float) densityDpi / DisplayMetrics.DENSITY_DEFAULT;
    }

In addition to the node, selector has also been changed to NonNull in some places in the UiObject2 file. We should pass proper selector for the 2.3.0 update, but such change will be large, thus I have updated only AccessibilityNodeInfo which is pretty safe as a PR here.
(I haven't investigated about the selector thing yet)

I have tested this code with an Android 11 real device. This code affects click for example as below code. It worked without issues

> e = driver.find_element id: 'com.android.chrome:id/url_bar'
=> #<Appium::Core::Element:0x..f88f6ee1501acc89e id="00000000-0000-00dc-ffff-ffff00000313">
> e.click
=> nil

An error with androidx.test.uiautomator:uiautomator:2.3.0 with null node when click occurred.

12-31 00:57:14.609 31495 31528 E appium  : Cannot create UiObject2 instance with 'null' selector
12-31 00:57:14.609 31495 31528 E appium  : java.lang.reflect.InvocationTargetException
12-31 00:57:14.609 31495 31528 E appium  : 	at java.lang.reflect.Constructor.newInstance0(Native Method)
12-31 00:57:14.609 31495 31528 E appium  : 	at java.lang.reflect.Constructor.newInstance(Constructor.java:343)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.appium.uiautomator2.model.internal.CustomUiDevice.toUiObject2(CustomUiDevice.java:100)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.appium.uiautomator2.model.internal.CustomUiDevice.getGestureController(CustomUiDevice.java:135)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.appium.uiautomator2.core.AxNodeInfoHelper.click(AxNodeInfoHelper.java:131)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.appium.uiautomator2.model.BaseElement.click(BaseElement.java:70)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.appium.uiautomator2.handler.Click.safeHandle(Click.java:37)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.appium.uiautomator2.handler.request.SafeRequestHandler.handle(SafeRequestHandler.java:59)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.appium.uiautomator2.server.AppiumServlet.handleRequest(AppiumServlet.java:277)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.appium.uiautomator2.server.AppiumServlet.handleHttpRequest(AppiumServlet.java:271)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.appium.uiautomator2.http.ServerHandler.channelRead(ServerHandler.java:77)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:435)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:250)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:266)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:611)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:552)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:466)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:438)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
12-31 00:57:14.609 31495 31528 E appium  : 	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
12-31 00:57:14.609 31495 31528 E appium  : 	at java.lang.Thread.run(Thread.java:923)
12-31 00:57:14.610 31495 31528 E appium  : Caused by: java.lang.NullPointerException: Attempt to invoke virtual method 'android.view.accessibility.AccessibilityWindowInfo android.view.accessibility.AccessibilityNodeInfo.getWindow()' on a null object reference
12-31 00:57:14.610 31495 31528 E appium  : 	at androidx.test.uiautomator.UiObject2$Api21Impl.getWindow(UiObject2.java:1061)
12-31 00:57:14.610 31495 31528 E appium  : 	at androidx.test.uiautomator.UiObject2.<init>(UiObject2.java:100)
12-31 00:57:14.610 31495 31528 E appium  : 	... 41 more

…x.test.uiautomator:uiautomator:2.3.0 update

app/src/main/java/io/appium/uiautomator2/model/internal/CustomUiDevice.java

mykola-mokhnach · 2023-12-31T16:13:41Z

I will create another PR later to avoid passing nullable selectors

…tor2-server into fix-for-uia-2-3-0

KazuCocoa · 2023-12-31T17:36:12Z

Thanks > selector
I saw a comment in the lib 2.2.0 that the selector variable in the uiobject2 was for debugging purposes, thus maybe nothing will break with the lib 2.2.0 with any values, but will affect in 2.3.0

mykola-mokhnach · 2023-12-31T20:42:36Z

app/src/main/java/io/appium/uiautomator2/model/internal/CustomUiDevice.java

+        //    }
+        AccessibilityNodeInfo accessibilityNodeInfo =
+                (node == null && android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.R)
+                        ?  new AccessibilityNodeInfo()


extra space here

Removed cd5692b
I used AS's config so it added these spaces automatically. I guess we could modify config file etc?

## [5.15.3](v5.15.2...v5.15.3) (2023-12-31) ### Bug Fixes * add an empty AccessibilityNodeInfo creation for future androidx.test.uiautomator:uiautomator:2.3.0 update ([#589](#589)) ([182d338](182d338))

github-actions · 2023-12-31T21:20:54Z

🎉 This PR is included in version 5.15.3 🎉

The release is available on:

Your semantic-release bot 📦🚀

KazuCocoa added 2 commits December 31, 2023 00:49

chore: add am empty AccessibilityNodeInfo creation for future android…

22f2a13

…x.test.uiautomator:uiautomator:2.3.0 update

Merge branch 'master' into fix-for-uia-2-3-0

828d53e

KazuCocoa requested a review from mykola-mokhnach December 31, 2023 08:52

KazuCocoa mentioned this pull request Dec 31, 2023

[Feat]: Support androidx.test.uiautomator:uiautomator:2.3.0 appium/appium#19597

Closed

KazuCocoa changed the title ~~chore: add am empty AccessibilityNodeInfo creation for future androidx.test.uiautomator:uiautomator:2.3.0 update~~ fix: add am empty AccessibilityNodeInfo creation for future androidx.test.uiautomator:uiautomator:2.3.0 update Dec 31, 2023

mykola-mokhnach reviewed Dec 31, 2023

View reviewed changes

app/src/main/java/io/appium/uiautomator2/model/internal/CustomUiDevice.java Outdated Show resolved Hide resolved

KazuCocoa changed the title ~~fix: add am empty AccessibilityNodeInfo creation for future androidx.test.uiautomator:uiautomator:2.3.0 update~~ fix: add an empty AccessibilityNodeInfo creation for future androidx.test.uiautomator:uiautomator:2.3.0 update Dec 31, 2023

KazuCocoa added 2 commits December 31, 2023 09:32

do not resign

a18299d

Merge branch 'fix-for-uia-2-3-0' of github.com:appium/appium-uiautoma…

ab59c8a

…tor2-server into fix-for-uia-2-3-0

mykola-mokhnach reviewed Dec 31, 2023

View reviewed changes

mykola-mokhnach approved these changes Dec 31, 2023

View reviewed changes

chore: remove spaces added by AS

cd5692b

KazuCocoa merged commit 182d338 into master Dec 31, 2023
17 checks passed

KazuCocoa deleted the fix-for-uia-2-3-0 branch December 31, 2023 21:19

github-actions bot added the released label Dec 31, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: add an empty AccessibilityNodeInfo creation for future androidx.test.uiautomator:uiautomator:2.3.0 update #589

fix: add an empty AccessibilityNodeInfo creation for future androidx.test.uiautomator:uiautomator:2.3.0 update #589

KazuCocoa commented Dec 31, 2023 •

edited

Loading

mykola-mokhnach commented Dec 31, 2023

KazuCocoa commented Dec 31, 2023

mykola-mokhnach Dec 31, 2023

KazuCocoa Dec 31, 2023

github-actions bot commented Dec 31, 2023

fix: add an empty AccessibilityNodeInfo creation for future androidx.test.uiautomator:uiautomator:2.3.0 update #589

fix: add an empty AccessibilityNodeInfo creation for future androidx.test.uiautomator:uiautomator:2.3.0 update #589

Conversation

KazuCocoa commented Dec 31, 2023 • edited Loading

mykola-mokhnach commented Dec 31, 2023

KazuCocoa commented Dec 31, 2023

mykola-mokhnach Dec 31, 2023

Choose a reason for hiding this comment

KazuCocoa Dec 31, 2023

Choose a reason for hiding this comment

github-actions bot commented Dec 31, 2023

KazuCocoa commented Dec 31, 2023 •

edited

Loading